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ABSTRACT 


The U.S. Navy transitioned to Computer-Based Training (CBT) in A and C schools in 
2003 after a 2001 Revolution in Training report claimed that the Navy would realize 
savings in cost and training time without negatively affecting the quality of sailors 
arriving to the Fleet. This thesis analyzes Operating and Support (O&S) cost data for the 
AN/SQQ-89(v) sonar system to detennine whether the transition to CBT contributed to 
increased Fleet maintenance costs. Detennining how actions to change one aspect of a 
system affects other areas of the system will provide insight for future decisions affecting 
O&S cost, system life cycles, and Fleet material readiness. 

The results of this thesis show that the conversion to CBT was not the sole 
contributing factor to increased Fleet maintenance costs or degraded Fleet material 
readiness. Changes to the Navy’s training, maintenance, and manning programs during 
the early 2000s were all contributing factors. 
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EXECUTIVE SUMMARY 


Changes to the U.S. Navy’s training, maintenance, and manning policies in the 
late 1990s and early 2000s resulted in degraded Fleet material readiness. One of the 
changes was the transition from Instructor Led Training (ILT) to Computer Based 
Training (CBT) in A and C schools in 2003. The US Navy expected significant cost 
savings as a result of this change without any degradation in the level of knowledge and 
skills produced by the training. This thesis examines the transition to CBT from a 
systems perspective to determine if it may have contributed to a loss of readiness and an 
increase in total system life cycle costs. The thesis looks at several questions. First, did 
the use of CBT in A and C schools result in sailors entering the fleet with lower levels of 
knowledge and skill sets than those sailors completing ILT? Second, did the transition to 
CBT result in cost savings in training? And finally, from a systems perspective, if sailors 
trained with CBT had lower knowledge and skill levels, did this contribute to increased 
operations and maintenance (O&M) costs? 

To answer the first question, government studies of the Navy’s CBT training were 
examined, which confirmed that while the transition to CBT resulted in shorter training 
times, sailors reporting to the Fleet were not as well prepared as ILT-trained sailors of the 
past. To answer the second question, DON Budget Reports from FYs 2000 to 2012 were 
analyzed to understand changes in training costs during the periods before and after the 
implementation of CBT. The analysis reveals that overall training costs remained fairly 
flat and specialized skills training costs nearly doubled over the same period, which 
suggests that any cost savings were realized in areas other than training delivery in A and 
C schools. 

To answer the final question, the thesis examines changes to training and 
maintenance costs from both a macro and micro perspective. At the macro level, analysis 
of DON O&M budgets for the periods before and after CBT implementation shows that 
Intermediate Maintenance budgets more than doubled and Depot Level Maintenance 
budgets nearly doubled during the period of interest. Unfortunately there were too many 
confounding variables that could have affected O&M costs during this period of time to 



draw any conclusions about the effect of training on maintenance costs. To control some 
of these variables it is necessary to examine the question at a micro level. 

This thesis focused on the AN/SQQ-89(v) sonar system to understand how the 
conversion to CBT affected one system in particular. Reports on the Navy’s changes in 
training, maintenance, and manning were examined in detail to understand what changes 
were made and what the effects of those changes were on training, readiness, and cost. 
Additionally, Operating and Support (O&S) cost data and 89 system data were analyzed 
to detennine whether the conversion to CBT had a direct impact on rising maintenance 
costs for the system. Data analysis revealed that there was not a direct correlation 
between the conversion to CBT and increased maintenance costs for the AN/SQQ-89(v) 
sonar system. While training costs decreased and maintenance costs rose in the 2000s, the 
events were not directly related. It is more likely that massive systemic changes in 
training, maintenance, and manning caused unintended second- and third-order effects 
that collectively contributed to degraded Fleet material readiness. 

For instance, the way that CBT was implemented in Navy A and C schools 
produced sailors that were ill-equipped to perform basic rate-related tasks. At the same 
time, the removal of instructors from the schoolhouse who were subject-matter experts 
(SMEs) in the course they were teaching exacerbated the problem. Additionally, SME 
instructors would typically return to the fleet after their teaching tour and provide 
valuable experience to their next ship. Since instructors were removed from the 
schoolhouse, this important shipboard training resource was eliminated. Since Fleet 
manning was reduced, there are fewer sailors available to perfonn shipboard 
requirements. Since sailors are not as technically competent as their predecessors, and 
there are fewer sailors available, material readiness suffers. At the same time, the 
elimination of enlisted shore billets and Shore Intermediate Maintenance Activities 
(SIMAs) caused degradation in sailors’ technical competence and created a backlog of 
jobs, further degrading Fleet material readiness. 
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I. INTRODUCTION 


A. BACKGROUND 

As the United States Navy entered the new millennium, Navy leadership 
expressed concern that the training programs in place would not adequately meet future 
training demands (Executive Review of Navy Training (ERNT), 2001). The Defense 
Science Board (DSB) Task Force stated: 

Future training must be delivered to the individual, to units, and to joint 
forces, when it is needed, not in the schoolhouse after which there is time 
for proficiency to decay. Training must be applied over and over again as 
the composition of the units and joint forces changes and as skills erode 
over time. Training must also become an integral part of the acquisition of 
hardware or we will fail to achieve the perfonnance in our weapons 
systems that our other superiority (technology) strives to deliver (Chatham 
& Braddock, 2000, p. 3). 

At the time, rate training (specialized skills training) in ‘A’ and ‘C’ schools was 
primarily conducted through fonnal “schoolhouse” training, i.e., training that took place 
in a traditional classroom with an instructor present. In their 2000 Training Superiority & 
Training Surprise report, the DSB Task Force reported that once a skill is learned, 
proficiency in that skill can be forgotten if not used and exercised frequently (Chatham & 
Braddock, 2000). The skills taught in ‘A’ and ‘C’ skills can suffer from “learning decay” 
by the time sailors reach the fleet and can decay further if not reinforced by refresher 
training. The DSB Task Force called for a “second revolution in training” that would 
deliver training “to the individual, to units, and to joint forces (Chatham & Braddock, 
2000).” As a result, the Chief of Naval Operations (CNO) chartered an Executive Review 
of Navy Training (ERNT) to review the Navy training system and recommend solutions 
to improve training effectiveness and meet future training demands (ERNT, 2001). 

In 2001, ERNT released Revolution in Training: Executive Review of Navy 

Training Final Report (ERNT, 2001). According to the report, training activities 

comprised 14% of the overall US Navy budget. Additionally, formal schoolhouse 

training represents a large monetary investment due to the need for facilities, instructors, 

and laboratories, in addition to student costs. Furthermore, the report stated that future 
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training demands would outstrip the number of billets available under the legacy 
schoolhouse system, and the suggestion was made that the use of technologies and 
modem training methods would contribute to a training cost reduction without producing 
a reduction in the quality of sailor produced by the training. 

As a result of the 2001 report, the U.S. Navy implemented a new training strategy 
that used several of the report’s suggestions. Task Force EXCEL (Excellence through 
Commitment to Education and Learning) was established under the leadership of the 
CNO, Admiral Vem Clark. The goals of Task Force EXCEL were to establish a 
continuum of lifelong learning, use a streamlined funding process and a single training 
authority, create a Human Performance Systems Model (HPSM), and to link training and 
acquisition (Naval Personnel Development Command, 2002). 

Part of this new strategy included the addition of new training technologies such 
as distributed learning, CBT, collaborative learning, and computer-mediated learning. 
The Navy claimed that the introduction of CBT would reduce both training time and 
training costs without reducing the quality of training received (ERNT, 2001). 

B. SCOPE 

This thesis looks to answer several questions. First, did the use of CBT in A and 
C schools result in sailors entering the fleet with lower levels of knowledge and skill sets 
than those sailors completing ILS? Second, did the transition to CBT result in cost 
savings in training? And finally, from a systems perspective, if sailors trained with CBT 
had lower knowledge and skill levels, did the conversion to CBT contribute to increased 
operations and maintenance costs? 

A 2009 Naval Inspector General (IG) Report, Computer Based Training, reported 
that the introduction of CBT did reduce training time. However, sailors arriving to the 
fleet under CBT did not usually meet the required Knowledge, Skills, Abilities, and Tools 
(KSATs) upon reporting on board. Because of this, ships had to take the time to train 
sailors up to acceptable standards (Naval Inspector General, 2009). The analysis of 
Department of the Navy (DON) Budget Reports conducted as part of this thesis research 
shows that the amount of money spent on training remained relatively consistent from 
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FYs 2000 through 2010. This suggests that the transition to CBT has not produced the 
desired reduction in costs. Furthermore, analysis of the DON O&M budgets the periods 
before and after CBT implementation shows that Intermediate Maintenance budgets more 
than doubled and Depot Level Maintenance budgets nearly doubled during the period of 
interest. This macro level analysis confirms that overall DON costs have increased, but 
does not provide enough granularity to answer the final question, namely, “if sailors 
trained with CBT had lower knowledge and skill levels, did this contribute to increased 
operations and maintenance costs?” A closer examination of a particular system (the 
system selected was the AN/SQQ-89(v) sonar) was the method used to answer this 
question. 

In order to answer these research questions, this report starts with a discussion of 
the Navy’s legacy training system, the Revolution in Training, and CBT. Next, the thesis 
discusses the Navy maintenance process, including Depot Level Maintenance, 
Intennediate Maintenance (IM), and the realignment of maintenance facilities during the 
2000s. Finally, an examination of the DON budgets from FYs 2000 through 2010 is 
conducted, paying particular attention to training and maintenance figures. 

At the macro level, there are many variables besides CBT that could contribute to 
rising maintenance costs, including the Global War on Terror (GWOT) and increased 
operations tempo (OPTEMPO). However, it is impractical to isolate the impact of CBT 
on Navy maintenance costs at the macro level. Instead, it is necessary to look at the 
impact of CBT on a particular system, program, or technology. This research effort 
focuses on a single system, the AN/SQQ-89(v) sonar, collecting data at a level of detail 
that allows for the control of the various variables that might impact maintenance costs. 
Thus, the scope of this thesis will include an examination of the AN/SQQ-89(v) sonar 
system to analyze how the conversion to CBT affected this system in particular. 
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II. TRAINING 


The U.S. Navy was interested in examining changes in its training structure in the 
late 1990s and early 2000s. The results were a 2001 Revolution in Training report that 
made several recommendations to change the Navy training system. This chapter 
examines the Revolution in Training report and two follow-on reports, Computer Based 
Training and Military Readiness: Navy Needs to Reassess Its Metrics and Assumptions 
for Ship Crewing Requirements and Training, that reported on the impact of the changes 
to Navy training 10 years later. 

A. INTRODUCTION 

Training in the Navy occurs throughout a sailor’s career, beginning with recruit 
training and ultimately ending with the Transition Assistance Program (TAP), when a 
sailor separates or retires from service. In addition to these events, sailors can expect to 
go through several other training programs throughout their career. Upon completion of 
recruit training, sailors are sent to specialized skill training in their designated job 
specialty, or rating. 

In-rate training begins in A school, where sailors learn the particular skills 
specific to their job. From there, a sailor can receive additional training in C school, as C 
schools provide specialized training in a particular aspect of a rating. For example, in the 
Sonar Technician—Surface (STG) rating, a sailor can earn the 0510 Navy Enlisted 
Classification (NEC), designating them as an AN/SQS-53D Sensor Subsystem Level II 
Technician/Operator (Navy Personnel Command, 2012). Once a sailor is assigned to a 
ship, he or she receives training for collateral duties such as quarterdeck watches, anti¬ 
terrorism/force protection (AT/FP) watches, weapons handling, and the at-sea fire party. 
Additionally, sailors can expect to receive general military training (GMT) in topics 
ranging from electrical safety to suicide prevention. 

Prior to 2003, nearly all of this training was conducted using some form of ILT 
(ERNT, 2001). Portions of recruit training were conducted in a fonnal schoolhouse 
setting, and all in-rate training was conducted in this manner as well. GMT was 
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conducted at command-level training “stand-downs,” a tenn which describes training 
occurring when designated personnel provide the GMT to the crew in a lecture format. 

After the 2001 ERNT report and the establishment of Task Force EXCEL, the 
Navy introduced several electronic training initiatives aimed at revolutionizing the 
training process and placing less emphasis on ILT. This section will define ILT and CBT 
as they pertain to the Navy, further discuss the Revolution in Training report, and discuss 
the impacts of Task Force EXCEL’s initiatives by examining a 2009 Navy IG report, 
Computer Based Training, and a 2010 Government Accountability Office (GAO) report, 
Military Readiness: Navy Needs to Reassess Its Metrics and Assumptions for Ship 
Crewing Requirements and Training. Additionally, this report will focus solely on ILT 
and CBT as they pertain to the specialized skill training received in A and C schools, 
considering carefully the claims which suggest that CBT would offer significant cost and 
time savings without degrading training quality. 

B. INSTRUCTOR-LED TRAINING (ILT) 

Until the early 2000s, in-rate training in the Navy was conducted in a fonnal 
schoolhouse setting, where instructors delivering the training are subject matter experts 
(SMEs) on the material they are teaching (ERNT, 2001). Typically, they come from the 
Fleet and have worked on the equipment they are teaching about. Information is delivered 
in the form of lectures and instructors are able to supplement the lecture material with 
tips and anecdotes from their career experiences (Naval Inspector General, 2009). 

In addition to lectures, sailors are able to reinforce their comprehension of the 
lecture material through hands-on experience in a laboratory setting. In maintenance 
courses, students are able to work on the exact equipment they see in the Fleet and 
instructors are able to simulate equipment casualties for technicians to troubleshoot. 

Instructors are able to tailor the delivery of material to a class based on the 
students’ level of comprehension. For example, if a class has difficulty understanding a 
particular concept, the instructor can choose to spend more time in the lab to reinforce 
what is learned during the classroom portion. 
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There are several benefits to ILT. Since a single instructor teaches a large group 
of students, group learning techniques can be employed that would otherwise be 
unavailable in one-on-one or CBT instruction. The fonnation of small groups within a 
class fosters team-building and allows students to help and teach each other. Compared to 
the costs of software development, testing, and hardware purchase, ILT is in some ways 
more cost-effective, depending on class size and length of use. Additionally, the 
controlled classroom environment offers fewer distractions than CBT or distance 
learning. Finally, ILT doesn’t take as long to develop as CBT. It takes approximately 34 
hours to develop one hour of ILT (Chapman, 2007), while it takes approximately 220 
hours to develop a standard e-leaming course (Chapman, 2006). 

ILT also has its disadvantages. Since everyone has different learning capabilities, 
some students may be more advanced and become bored while waiting for slower 
learners to catch up. Conversely, slow learners may have difficulty keeping up. 
Depending on the size and duration of the course, ILT may be more expensive than CBT. 

C. COMPUTER-BASED TRAINING (CBT) 

Computer Based Training defines CBT as “individual or group self-paced 
instruction using a computer as the primary training medium, to include web-delivered 
Navy E-Leaming (NEL)” (Naval Inspector General, 2009, p. ii). Based on the 
recommendations of the Revolution in Training Report, the Navy established Task Force 
EXCEL and brought CBT into the schoolhouse at A and C schools. By the time of 
release of the 2009 Navy IG report, 34% of all A school training was delivered by CBT 
(Naval Inspector General, 2009). 

In the CBT learning environment in A schools, students go through learning 
modules on a personal computer (PC). Since students are at their own work stations, the 
learning is self-paced. When a student is done processing the infonnation presented on 
the screen, they click “next” to proceed to the next piece of infonnation. There are 
usually small knowledge assessments throughout the module, followed by a final 
knowledge assessment at the end of the module (Naval Inspector General, 2009, p. 7). 
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Because the learning is self-paced, instructors were replaced with “facilitators.” 
Facilitators are not necessarily SMEs in the subject matter being delivered in the CBT 
modules. The purpose of the facilitator is “to ensure classroom rules are followed, assist 
with computer-related issues, and monitor student progress. They do not provide 
reinforcement of learning objectives or enhance retention of course material” (Naval 
Inspector General, 2009). The problem with replacing instructors with facilitators is that 
students cannot go to a facilitator with a question about subject material. It removes the 
opportunity to teach when a student is confused. Additionally, instructors provide an 

example of what an experienced sailor should be in terms of appearance, demeanor, and 
professionalism. This was lost in the switch to facilitators (Navy Inspector General, 
2009). 

There are several advantages to CBT. The learning is self-paced and if the course 
is offered as distance learning, the schedule to take the course is flexible. Students can 
complete the course at their own pace, which generally shortens training time. Since there 
are no instructors involved, the message doesn’t change from one person to the next 
(Dhanjal & Calis, 1999). 

CBT also has disadvantages. Students don’t have the ability to talk to a SME if 
there is confusion about a topic. Since the schedule is flexible, procrastination can set in. 
Since the training is one-on-one, students don’t have the benefits provided by group 
instruction. 

D. REVOLUTION IN TRAINING: A KEY REPORT ON NAVY TRAINING 
1. Background 

In order to provide an adequate context for the data analysis conducted in 
subsequent chapters of this thesis, an intensive examination of a key document explaining 
training policy in the Navy is necessary. This section features a summary of crucial 
infonnation provided by a CNO initiated study into how training is conducted in the U.S. 
Navy. 

In October 2000, the CNO tasked the fonnation of a group to conduct a review of 

Navy training, i.e., the Executive Review of Navy Training group. The ERNT group was 
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charged with providing insights on how to improve and align training organizations, 
leverage civilian training practices, and use new technologies to provide a continuum of 
training for sailors. The 24-member group was comprised of military and civilian 
personnel, members of academia, research institutions, and industry. 

During their review, the group noted that the demands for training had increased. 
At the macro level, the training demands are driven by the set of Required Operational 
Capabilities and Projected Operating Environments (ROC/POE). ROC/POE is used to 
detennine specific warfighting missions for each ship. Training requirements are derived 
from these missions and are then used to determine specific training requirements for 
sailors. Changes in the ROC/POE led to increased training requirements for units during 
Inter-Deployment Training Cycles (IDTCs). For example, IDTC requirements for an air 
squadron increased by 31% over an 8-year period. Ship training requirements are passed 
down to the sailor level, and in many cases, this resulted in billets which could not be 
filled because there were no sailors with the required training to fill them. In the 
Infonnation Systems Technician (IT) rating for example, billet shortfalls ranged from 19- 
79% for certain NECs. At the same time, the pool of experienced sailors had decreased 
due to the 1990s drawdown and retirements from that group of remaining sailors, and the 
fact that it was difficult to compete with the civilian sector and a healthy U.S. economy. 
Many sailors were receiving training through the military and then leaving for jobs in the 
civilian sector. 

The group reported that the Navy training system was not constructed to support 
the training demands based on increased ROC/POE requirements. The additional training 
requirements are tied to NECs and billets which require training in schoolhouses. Since 
there a finite number of seats available in the schoolhouses, demand outstrips the number 
of available seats. Because of this, there were gaps in what types of training (NECs) that 
current and/or potential sailors wanted and what could be delivered. The group suggested 
that technology and the science of learning offered several opportunities to improve the 
Navy training system by reducing training time through CBT and offering distributed 
learning opportunities that could be executed at the workplace. This is discussed further 
at the end of the report, (in the Conclusion/Recommendations section, II.D.6.) 
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2 . 


Data 


During their study, ERNT collected data reflecting the status of Navy training in 
2000. They found that the US Navy training budget in 2000 was $10B, which accounted 
for 14% of total annual funding. Funding from the 2000 training budget went to more 
than 1,000 training sites. From these sites, more than 900,000 training opportunities were 
provided to approximately 460,000 active duty and reserve sailors (ERNT, 2001). But the 
study found flaws in the way training was funded. For example, training money came 
from 11 different resource sponsors and was then distributed to 12 training claimants who 
spent the money. This lack of organization is inefficient, since money being spread 
through so many areas causes some training areas to be redundantly funded, while others 
are neglected (ERNT, 2001). 

In one part of the study, ERNT noted that e-learning reduced training costs by 30- 
60% while in another part of the study they noted that the use of technology saves the 
Navy 25-33%. ERNT claimed that e-leaming reduced training time by 20-40% while 
increasing knowledge retention by 10-30%. The report cited a U.S. Anny example that e- 
leaming resulted in $155M in cost savings and a 30% reduction in training time when e- 
leaming was implemented for 525 courses (ERNT, 2001). These contradicting numbers 
illustrate how difficult it is to truly capture how the use of technology impacts training. 
Additionally, they found that there was no central authority in the training system 
(ERNT, 2001). The lack of authority causes ships to be assessed in the same training 
areas multiple times by different organizations. The lack of coordination places additional 
stresses on ships with an already full operational schedule. For example, the tactical 
training organization was made of 38 different organizations that performed management 
functions, 39 that could coordinate exercises, and more than 60 schoolhouses (ERNT, 
2001). They also discovered that there were 63 different organizations that could impose 
training requirements (ERNT, 2001). 

3. Findings 

In addition to the data gathered, ERNT made several observations about how the 
Navy training organization operated up to the point of the report (ERNT, 2001). The 
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Navy’s problems with training began at the recruitment phase. Recruiters tried to get 
people into the Navy, but they weren’t concerned with matching skills and interests to 
Navy billet requirements. Recruiting was more about meeting quotas than making sure 
the people were the right fit for the job they were assigned. Thus, recruiting practices 
weren’t aligned with training. The Navy’s recruiting philosophy thus contrasts negatively 
with modem industry, as modem industry views human resources and training as in 
integral part of their business strategy (ERNT, 2001). Modem industry places an 
emphasis on human performance and the science of learning. Human performance 
recognizes that how a human interacts with a piece of equipment is just as or more 
important than the sophistication of the equipment itself. While industry tends to value 
human perfonnance, many military programs put more emphasis on the equipment 
without considering how the human will interact with it (ERNT, 2001). The science of 
learning focuses on how individuals learn and recognizes that each person learns at a 
different speed. Researchers recognized that classroom instmction in the military is 
geared towards the slowest learner because everyone must have grasp of the topic before 
proceeding to the next one (ERNT, 2001). Self-paced CBT allows each individual student 
to proceed at his or her own pace, speeding up the learning process for faster learners and 
allowing slower learners to take as much time as necessary to grasp a subject. Industry 
and civilian research have highlighted this concept, and this was recognized as a need for 
the Navy (ERNT, 2001). 

In addition to the recruiting problem, there are several structural problems with 
the Navy training organization. There were too many commands and organizations with 
redundant training requirements and there was no coordination between the commands. 
For example, training funding was drawn from 11 different areas and distributed to 12 
claimants, so there was no central way to manage funding. Since there was no central 
management for the funding or administration of training, multiple funding areas and 
claimants led to redundancies and gaps throughout the training organization. Some 
training organizations had overlapping interests, so some training areas were funded 
multiple times. At the same time, some training areas fell through the cracks because 
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there was no oversight to tell whether that training area had been met or funded by one of 
the multiple training organizations. 

Finally, it was discovered that human systems integration (HSI) did not receive 
enough consideration during acquisition, revealing that personnel were not properly 
regarded as important contributors to system perfonnance. Human performance as an 
area of study encompasses not only equipment, but also the skills required to operate a 
particular system. If human performance is considered early, developers can detennine 
what skills are required, how much training is required, how many operators are required, 
or if a system will require redevelopment. Often, since human performance was not 
considered early in the design process, the need for additional manning on ships arose. 
The money required to provide the additional manning typically came from training and 
Integrated Logistic Support (ILS) funds for the system (ERNT, 2001). 

Reviews of the training program revealed that ILT was the primary training 
method used to deliver training. Interviews with sailors revealed that their best training 
experiences occurred when knowledgeable instructors delivered relevant content and 
provided hands-on training (ERNT), 2001). Upon reporting to a ship, sailors felt that they 
did not meet the level of knowledge requirements when reporting to their new commands 
and saw several obstacles to receiving additional training. These impediments included a 
lack of time to train, on-board personnel shortages, course availability, inadequate 
facilities or equipment, and a low priority level set by their command to receive 
additional training (ERNT, 2001). 

In addition to the sailor’s noted concerns, there has also been an increase in 
mission sets added to ships, i.e., there were an increase in the amount of training 
requirements assigned to sailors, but no additional sailors were added to the ship. A final 
concern is that throughout a ship’s training cycle, training and knowledge retention 
atrophies through the course of a deployment due to learning decay (ERNT, 2001). 

The additional requirements made it difficult for sailors to receive OJT. But a 
clear recommendation from the study was the need for and the undisputed quality of OJT: 
this is stated boldly by the research group: 
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The Navy needs to provide explicit support for the conduct of on-the-job 
training. OJT is, by all accounts (including our interviews with Sailors), 
the most effective training that our Sailors experience. There is great 
potential to improve that training by focusing on it explicitly during the 
implementation of the recommendations of the Executive Review of Navy 
Training, (pp. 58-59.) 


Unfortunately, it is difficult to detennine what training tools were the best to use. 
This is best explained by a statement made earlier in their report which indicates that the 
above statement may not be as easy to implement as it might seem; as the report had 
observed in another place, 

Historical evidence says that if the training system does not send 
adequately trained Sailors to the Fleet, they will not ‘catch up’ in the Fleet 
—or in actual war fighting. In fact, operational perfonnance is extremely 
sensitive to readiness levels at the onset of operations or combat. This is 
well documented (by early mission attrition data) from World War II, 

Korea, and Vietnam, and the lesson has been relearned in the recent Desert 
Fox and Noble Anvil operations. (ERNT, 2001, p. 6.) 

The importance of adequate training is clear, because of the fact that without the 
necessary foundation, sailors can’t make up for poor training with training occurring at a 
later date. 

An additional training problem to consider has to do with the creation of training 
plans, as some particular systems have problems from the beginning because of a poor 
conception. For example, all systems are required to have Navy Training Systems Plans 
(NTSP), which is described here: 

An NTSP defines the training necessary to support the operation and 
maintenance of the system. The NTSP process, when properly executed, 
provides opportunities for the Fleet, resource sponsor, CNET, and 
personnel command to review and assess training requirements. (ERNT, 

2001, p.10) 

Unfortunately, many systems were installed on afloat units without NTSPs. This 
was attributed to the rapid fielding of systems using commercial off-the-shelf (COTS) 
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and government off-the-shelf (GOTS) technologies. COTS/GOTS systems were usually 
upgrades to existing systems and in many cases, NTSPs were never sent. This presents a 
challenge for training organizations because training material is either unavailable or out 
of date, depending on whether the system is new or updated. In both cases, training 
commands and fleet units have to play catch-up once a system is fielded without a NTSP. 

4. Observations 

Some key points from the 2001 analysis of the state of Navy training were that 1) 
training may not continue to meet readiness requirements in the post-Cold War era given 
the decreased manning and increased requirements imposed upon ships and 2) due to a 
good job market in the civilian sector at the time (2000), it was possible that the Navy 
would be unable to recruit the right number of people with the proper skillsets. 

The important recommendations from the ERNT for the purposes of this thesis 
research were that the Navy’s training program should be realigned to meet the following 
objectives: 

1) Sailors should be trained to maintain and operate systems at the highest levels 
of readiness and performance, 

2) Training considerations should begin at equipment acquisition and continue 
through operation of the system until retirement, 

3) Sailors must be equipped with the necessary skillsets to operate/maintain the 
equipment from cradle to grave, and 

4) The objective of training is to achieve readiness and ensure mission 
accomplishment (ERNT, 2001) 

A key point in considering CBT involves the question of how to conduct training 
with respect to new equipment. While the assumption has been that when a new piece of 
equipment was deployed, fonnal schoolhouse training was required, it has been 
suggested that that schoolhouse training was expensive and that training facilities 
represented a large investment over a long period of time. Because of the expense of 
training, and because as of 2000, the demand for billets in Navy schoolhouses exceeded 

14 



the budgeted forecast, resulting in a shortfall of qualified sailors in the Fleet, it has been 
proposed that modern learning techniques such as CBT would reduce the need to send 
students to the schoolhouse (ERNT, 2001) One key topic for analyzing training is the use 
of performance measures in place to assess the effectiveness of training. The Navy did 
not have a method to assess whether training was valuable, in contrast to civilian 
industry, which was developing measurement criteria for training because, in their view, 
training was tied to earnings. One way to measure performance is the 1959 Kirkpatrick 
4-level measurement model that is the industry standard. In a 2005 survey, 96% of 
companies interviewed used some elements or variation of the Kirkpatrick model (Bassi, 
Gallager, & Schroer, 1996). The Kirkpatrick model includes the following measures: 

• Reactions - Studies students’ answers to questions about the training they 
received. 

• Learning - Measures whether the student retained the skills taught. 

• Behavior - Measures whether the learner acquired the skills taught 
through a demonstration test. 

• Results - Measures “whether or not the organization actually achieves its 
desired objectives as a function of training.” (ERNT, 2001) 

From these observations, the ERNT group developed a series of conclusions and 
recommendations to improve Navy training. 

5. ERNT Report Conclusions and Recommendations 

Intensive analysis of Navy training recommended a 4-step process to 
revolutionize Navy training: 1) Adopt a Navy Human Perfonnance System Model 
(HPSM), 2) Align training, 3) Emphasize human perfonnance in acquisition, and 4) 
Establish a lifelong learning continuum (ERNT, 2001). The next section of the thesis 
addresses each of these four steps. 

a. Adopt a Navy HPSM 

The Navy training system has not been not well organized. For example, 
there are several organizations with overlapping requirements and there was no 
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communication between training entities. There was not a feedback system in place to 
detennine whether training was meeting requirements or improved job perfonnance. 
ERNT proposed realignment of Navy training using a systems-level approach under the 
HPSM. The proposed HPSM used a 4 quadrant process which is rooted in the basic 
concepts of the Systems Engineering process: 

• Define Requirements (Ql) 

• Design Solutions (Q2) 

• Develop, Build and Integrate Tools (Q3) 

• Execute and Measure Effectiveness (Q4) (ERNT, 2001, p. 33.) 

Requirements definitions (Ql) is centered on breaking down jobs and job 
tasks into behaviors and competencies, then ranking them on importance to fonn job 
performance standards. The requirements should be different for different stages in a 
career. Design of solutions (Q2) incorporates the science of learning and human 
performance sets. Analysts and SME take the requirements developed in Ql and 
determine the best methods to meet them. In addition to solution options, performance 
metrics and cost analysis should be considered. Once a solution option is selected, the 
choice should be developed (Q3). The solution option can include any mix of formal 
classroom instruction, CBT, distributed learning, manpower adjustments, job 
performance aids, job re-design/automation, etc. (ERNT, 2001) 

If training is the solution that is accepted, training occurs and is measured 
in Q4. If multiple organizations are involved in the training it should be integrated so 
steps are not duplicated. The measurement should include 4 levels of feedback: 

• Is the student enjoying the experience? 

• Is the student learning facts? 

• Is the Sailor more productive in prescribed tasks? 

• Is the team/command more proficient because of the perfonnance 
enhancement solution? (ERNT, 2001, p. 35.) 
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The recommendation for meeting Navy training objectives involves 
integrating training technology, human perfonnance requirements, and the science of 
learning into the HPSM and melding it into a framework for determining solution options 
for perfonnance issues. From the study of the science of learning, it is recognized that 
individuals respond differently to different techniques, learning is improved when 
tailored to the individual; retention is improved when more than one learning method is 
used, and the transfer of learning is more effective when supported by feedback, support, 
and practice (ERNT, 2001). Thus, learning is most effective when it features multiple 
methods, and the Navy Learning Model includes instructor-led training, reference-based 
learning, computer-mediated learning, and collaborated learning with reinforcement 
through OJT and mentoring (ERNT, 2001). 

To improve Navy training, the establishment of a Human Perfonnance 
Systems Organization (HPSO) to design near and long-tenn solutions has been 
recommended. The HPSO would handle the QI and QII functions of the HPSM to 
include: 

• Apply Science of Learning and Human Perfonnance considerations to 
Navy job requirements 

• Develop perfonnance measures 

• Generate knowledge, skill, abilities & tools (KSATs) 

• Assess the cost effectiveness (e.g., return on investment) of 
perfonnance and learning solutions 

• Develop and maintain Navy-wide strategic learning plans (simulators, 
e-Learning, virtual reality, CBT, etc.) (ERNT, 2001). 

b. Training Alignment 

ERNT recommended establishing an integrated training organization to 
eliminate redundancy and maximize resources. The proposed organization would collect 
perfonnance data and results, recommend improvements, assess cost effectiveness and 
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return on investment (ROI), control all training activities and processes, manage all 
schoolhouse training, and have its own funding (ERNT, 2001). 

Additionally, training could be aligned at the OPNAV level. Training 
requirements were fragmented across several OPNAV branches. At the OPNAV level, 
the training re-organization would be responsible for Navy-wide training policies, 
establish training requirements processes, and establish and monitor a training 
perfonnance tracking system (ERNT, 2001). 

c. Emphasize Human Performance in Acquisition 

The group recommended 1) Considering training and human performance 
in the design and acquisition of systems, 2) Holding program managers accountable and 
ensure training is considered early in the design process, 3) Developing a Navy 
instruction that guides the development of navy training systems, and 4) Utilizing HSI in 
systems development (ERNT, 2001). 

d. Establish a Lifelong Learning Continuum 

ERNT believed that establishing a lifelong learning continuum would 
improve performance and provide better use of “human capital.” Additionally, they 
placed an emphasis on blended learning solutions and encouraged the use of learning 
tools such as the Internet. ERNT’s proposals relied on the idea of an “empowered 
individual learner,” who would be personally responsible for his/her own learning, would 
be a creative participant in the training process, would achieve proficiency in their job, 
would learn throughout their career, and enjoys the career training process (ERNT, 
2001 ). 

E. 2009 NAVY IG REPORT 
1. Background 

The Navy IG commenced a review of CBT in the Navy after hearing concerns 
from the Fleet about the level of knowledge of sailors reporting to ships from A schools. 
The IG conducted a thorough review of Navy training as it related to CBT by conducting 
interviews and studies at all levels of the training continuum—policy makers, 
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schoolhouses, students, and afloat units. The goal of the IG report was to answer the 
question, “Does the Integrated Learning Environment (ILE), as supported by CBT, meet 
its goal of delivering a trained and ‘Sailorized’ asset to the Fleet?” (Naval Inspector 
General, 2009, p. 1) The report included a review of existing training policy, assessed 
effectiveness of that policy, and identified areas of improvement while offering potential 
solutions. The policy review included a review of the 2001 ERNT Revolution in Training 
report (Naval Inspector General, 2009). 

2. Findings 

The IG report found deficiencies in 11 key areas, several of which related to the 
Revolution in Training report or changes that were implemented because of it: 

a. Curriculum Design and Development 

The ILE was established as a result of the Revolution in Training Report. 
The purpose of the ILE is to establish an infrastructure for electronically-delivered 
training material and to provide an administrative and technical repository for the 
distribution, storage, and maintenance of Navy learning material (Naval Inspector 
General, 2009). In order for a course to be delivered via CBT, Department of Defense 
(DoD) Directive 1322.18 Military Training, states that electronic training must be the 
first alternative when choosing the most cost-effective training solution. The directive 
also states that DoD members must receive the most effective training possible 
(Department of Defense, 2009). These statements imply that some sort of analysis of 
alternatives (AOA) must be conducted when determining the optimal medium of training 
delivery. 

The IG report revealed that there were no standards for the development of 

CBT material. The quality of CBT material varied depending on the program sponsor, so 

different pieces of equipment have different types and quality of CBT material, meaning 

that the effectiveness of the material could be highly variable. There was little evidence 

to show that SMEs had input in the development or testing of CBT material. In many 

cases, old lecture material was quickly converted into CBT format, but there was not a 

process in place to verify the accuracy of the information. In some cases, assessment tests 
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had incorrect answers, and facilitators developed workarounds to enable students to pass 
the tests (Naval Inspector General, 2009). 


In addition to the development of the CBT material, another problem was 
that more than 30% of all CBT material did not reside on the ILE. In the case of SECRET 
coursework, the Secret Internet Protocol Router Network (SIPRNET) structure was not 
fully developed, forcing schoolhouses to maintain SECRET CBT coursework on local 
SIPR servers (Naval Inspector General, 2009). 

There was very little evidence to suggest that any AOAs were conducted 
to determine whether CBT was the appropriate choice for any of the CBT modules. Also, 
there did not appear to be any standardization for development, testing, and funding of 
CBT materials as specified in the ERNT report. 

b. Life Cycle Management (LCM) 

DoD Instruction 1322.20, Development and Management of Interactive 
Courseware (ICW) for Military Training, states that DoD Component Heads are to 
“Provide life-cycle management (LCM) for their Interactive Courseware (ICW) 
programs” (Department of Defense, 2004). The IG found that LCM for CBT programs 
usually did not account for program and curriculum updates. Contrary to what ERNT 
expected, these problems resulted in program updates taking more time to take effect than 
traditional ILT. Facilitators were forced to conduct ILT or develop workarounds in the 
CBT programs (Naval Inspector General, 2009). 

c. Delivery Systems for Web-Based Technology 

In A schools, CBT is conducted in an electronic classroom (ECR) 
environment. The ECR consists of several PCs connected to a server or a fann of servers. 
Student progress, exam scores, and transcripts are managed on a Learning Management 
System (LMS) that resides on the same server as the CBT it supports (Naval Inspector 
General, 2009). 

As of the 2009 report, more than 26,000 computers were li nk ed to ECRs. 
The average age of the computers was approximately 6 years (Naval Inspector General, 
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2009, p. 5). The IG group found that most ECRs needed new equipment and the old 
equipment caused several problems for students due to equipment malfunction, low 
bandwidth, and poor infrastructure. Students expressed frustration with the ECRs because 
of these problems. They reported that it created a distracting environment that disrupted 
their learning opportunities. 

Since ECRs are managed by different program sponsors depending on the 
system, the quality of LMS varied from site to site (Naval Inspector General, 2009). The 
LMS was designed to reduce administrative burden on facilitators, but in some cases, 
they doubled or tripled time spent on administrative duties due to poor construction. 

d. Electronic Instruction and “Generation Y” 

Revolution in Training highlighted the need to compete with modem 
industry to recruit talented personnel. ERNT discovered that modem organizations used 
e-leaming as a training tool and suggested that the future workforce Navy training system 
should capitalize on e-leaming technology to attract a savvier workforce. While it is true 
that “Generation Y” has grown up in the computer age and are technologically savvy, the 
IG found that it did not necessarily mean that they were more accustomed to learning on 
computers. Sailor interviews revealed that they preferred to use technology for 
entertainment and communication. Sailors were more familiar with ILT and were not 
accustomed to learning in front of a computer for an entire class-day (Naval Inspector 
General, 2009). 

e. Instructional Design 

The IG report cited modem learning theory, specifically the visual, 
auditory, and kinesthetic (VAK) model (Naval Inspector General, 2009). According to 
VAK theory, students learn mostly through sight, hearing, and touch, with one or two of 
the three methods being the most predominant. Proponents of the VAK model suggest 
that effective training should include all three elements in order to reach all styles of 
learners (MacKay, 2007). The IG found that most Navy CBT did not effectively capture 
all of the VAK elements. Typically, CBT always addressed visual learning, sometimes 
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addressed audio learning, but did not address kinesthetic learning at all (Naval Inspector 
General, 2009). 

CBT also did not effectively capture the four key elements of adult 
learning: motivation, reinforcement, retention, and transference (Lieb, 1991). Adult 
learners must have an incentive to learn such as a comfortable learning environment, a 
topic that reasonably challenges them, or a sense of importance about the material they 
are learning. Learners should be rewarded for positive outcomes and made aware of 
mistakes. They must be able to retain and apply the information they have learned when 
in the workplace (Lieb, 1991). These elements were neglected because Navy CBT 
focused on speed of completion, removed instructors from the classroom, offered no 
incentives, and removed redundant training (Naval Inspector General, 2009). Each of the 
above points will be explained in more detail in the sections to follow. 

f Speed of Completion vs. Knowledge Mastery 

Incentive to learn has been cited as a key element of adult learning. In 
Navy CBT, the only incentive offered is to complete the course as quickly as possible. 
Because of this, students click through the training programs as quickly as possible in 
order to get to the exam. Since there is no penalty for failure, students try to leam enough 
information to pass the exam and then “data dump” what they have learned immediately 
after (Naval Inspector General, 2009). If they fail the exam, they can go back and try 
again until they pass, so there is little incentive to actually leam. 

g. The Shift from Instructor to Facilitator 

In ILT, instructors were SMEs in the field they were teaching. Typically, 
they had just come from the fleet and had worked on the equipment that they were 
teaching on. They had operated equipment, maintained equipment, and restored 
casualties. Additionally, they were senior sailors and served as examples of how sailors 
should conduct themselves personally and professionally. If a student had a question 
about the material, the instructor could readily provide the answer and, most likely, had a 
personal “sea story” to offer that related to the material. Additionally, since ILT occurred 


22 



in a group environment, several sailors may have had the same question, so the exchange 
of information bene fitted the entire group. 

In the Navy CBT setting, however, this is not the case. Most facilitators 
are civilian contractors who were in the ECR to enforce classroom rules, monitor 
progress, and attend to computer problems. Active duty facilitators hold the 9502 
Instructor NEC, but they aren’t trained in CBT teaching (Naval Inspector General, 2009). 
Since CBT is done individually and is self-paced, facilitators respond to questions 
individually; however, since most facilitators aren’t trained in the topic, they are unable 
to answer the questions appropriately. Sailor interviews revealed student frustration with 
this issue (Naval Inspector General, 2009). Also, since answers are given one-on-one, 
there is no consistency from student to student. This is more of an issue with how CBT 
was implemented than a statement to the quality of the material contained within. It does 
illustrate, however, that there were several problems surrounding the CBT program that 
contributed to the problems experienced in the Fleet. 

h. Elimination of Redundancy 

The IG found that knowledge retention is related to repetition of material 
(Naval Inspector General, 2009). In ILT, information is taught orally and reinforced 
through practical application in laboratory settings. Since the goal of CBT is to reduce 
training time, repetition of material was eliminated. The result is that upon arriving to C 
school (after A school), students lack the basic knowledge required to begin the next 

course. In one example, the Aegis Training Readiness Center (ATRC) constructed an 
Awaiting Instruction Program (AIP) to re-teach material from A school in order for 
students to proceed on to C school (Naval Inspector General, 2009). 

i. Transference of “A ” School Training to the Fleet 

The lack of motivation, retention, and repetition in the Navy CBT 

environment leads to poor transference of skills when sailors arrive to the fleet. In 

addition to this, the IG report found that most CBT training was not linked to KSATs 

(Naval Inspector General, 2009). Because of this, sailors arrive to ships unaware of the 
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equipment they are working on or the tools they need to use. The IG noted that the 
expectations of the Fleet were not met by the training provided in the classrooms. Fleet 
leadership expects a “turnkey ready” sailor who arrives to the ship knowledgeable in their 
field and able to work with little training. In Fleet interviews, some commands reported 
that qualification time is nearly double what it was before the introduction of CBT (Naval 
Inspector General, 2009). The problem faced by the fleet is that either they take the extra 
time to train sailors properly and take time away from other important tasks, or they 
neglect the training to focus on operations. Either way, sailors and afloat units suffer. 

j. Training Pushed to the Fleet 

Sailors arriving to the fleet under CBT do not usually meet the required 
KSATs upon reporting on board. Because of this, ships must take the time to train new 
sailors up to acceptable standards. This task becomes difficult under current conditions 
due to increased OPTEMPO, increased training requirements, more complex equipment, 
reductions in manning, and the loss of additional sailors to fill Individual 
Augmentee/Global War on Terrorism Support Assignments (IA/GSA). Additionally, the 
shift from instructor to facilitator removed experienced sailors from divisions; sailors 
returning to the Fleet from instructor duty provided a valuable asset to a division (Naval 
Inspector General, 2009). 

3. Navy IG Report Conclusions 

The IG report noted that there were both positive and negative impacts of ERNTs 
recommendations to switch to CBT, and also recommended solutions. The report noted 
that training time had been successfully reduced, which resulted in cost savings in 
manpower and infrastructure (Naval Inspector General, 2009). 

Among the negative impacts the Navy’s implementation of CBT were that there is 

no centralized authority that governs standardization, development, content, contracting, 

or life cycle management of CBT. Also, CBT courseware isn’t aligned with the KSATs 

that sailors require in the fleet. There needs to be a method for afloat units to provide 

feedback to schoolhouses for course improvements. Finally, schoolhouses should shift 

back from facilitators to instructors (Naval Inspector General, 2009). Learning 
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environments that incorporate CBT with other teaching methods would allow instructors 
to return to the classrooms and provide the example that young sailors need. 

F. OTHER REPORTS 

Two reports, released in 2010 and 2011, addressed the results of the changes 
made in the Navy training system. These reports are summarized in this section. 

1. Military Readiness: Navy Needs to Reassess its Metrics and 
Assumptions for Ship Crewing Requirements and Training (GAO 
Report) 

In 2010, GAO released a report, Military Readiness: The Navy Needs to Reassess 
Its Metrics and Assumptions for Ship Crewing Requirements and Training at the direction 
of the House Armed Services Committee. The report examined the impacts of changes to 
both crew size and training. This portion of the thesis will examine the training portion of 
the GAO report. 

The GAO report noted that the Navy has made significant changes to the way it 
delivers training, as described in previous sections of this thesis. In assessing these 
changes, GAO found that the Navy had reported significant reductions in course length 
and cost but lacked data on student perfonnance. The Navy IG report highlighted the 
importance of tying CBT courses to KSATs and went on to note that fleet leadership felt 
that sailors did not meet the requisite KSATs upon reporting on board. GAO furthered 
this by commenting that the Navy has not established any quantifiable way to measure 
whether CBT has impacted a sailor’s KSATs once they report to a ship (GAO, 2010). 
The Navy did conduct one study in an attempt to compare ILT and CBT, but they 
examined only one instance of training and did not perform any follow-on assessments. 
The study was performed by the Center for Naval Engineering (CNE). CNE examined 
one class of engineering students. The class was divided into two groups, one that 
received training material via ILT and another that received CBT. CNE reported that the 
CBT group exhibited higher scores in level of knowledge (LOK) exams administered 
after the training (GAO, 2010). Unfortunately, no follow-on work was conducted to 
follow sailors to the fleet and re-test them at various time intervals. Since the study was 
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completed only with one test group, there is not enough data to examine adequately the 
validity of the results. Navy officials informed GAO that they would like to perform 
additional studies, but cited difficulties in coordinating with training entities and afloat 
units. Another difficulty in comparing ILT and CBT effectiveness is that there is no 
baseline ILT data available to compare with any CBT data that may be collected in the 
future (GAO, 2010). GAO findings on sailors’ level of knowledge (LOK) upon arriving 
to ships echoed the Navy IG reports that sailors were not arriving with sufficient 
comprehension of the KSATs for their job. Fleet interviews with sailors 11 afloat units 
revealed additional training time was required to bring CBT-trained sailors up to speed 
(GAO, 2010). The interviews also echoed Navy IG interview comments that cited the 
difficulties of providing the required OJT under reduced manning and increased 
requirements conditions. Some senior leaders reported that the LOK in new sailors was 
so low, that the senior leaders often stepped in to directly supervise, or even perfonn, 
maintenance actions that were previously routine for junior sailors (GAO, 2010). 

In some cases, commands resorted to establishing their own classrooms to bring 
newly-reporting sailors up to speed. For example, Afloat Training Groups (ATGs) in 
seven Fleet Concentration Areas (FCAs) established a 4-week Surface Warfare Officer 
(SWO) Introduction Academy that teaches newly-arriving officers basic division officer 
fundamentals required upon arrival to their ship (Jones, 2009). This was added as a 
supplement to “SWOS-in-a-box,” a set of training CDs that new officers are expected to 
complete when reporting on board, which was originally meant to replace the 6-month 
Surface Warfare Officer School (SWOS) that officers completed before reporting to their 
first commands (GAO, 2010). Based on their study, GAO recommended that the Navy 
develop performance metrics to assess the impacts of its training programs on readiness 
(GAO, 2010). 

2. Military Readiness: Navy’s Report to Congress on the Impact of 
Training and Crew Size on Surface Force Material Readiness (GAO 
Report) 

In 2011, GAO released Military Readiness: Navy’s Report to Congress on the 
Impact of Training and Crew Size on Surface Force Material Readiness in response to 
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DONs report to Congress of the same name. In the DON report, Navy officials conceded 
that, “eliminating some training courses and shifting to more computer-based training 
resulted in crews arriving on board their ships not fully ready to perform apprentice-level 
maintenance actions and contribute effectively to existing material demands” (GAO, 
2011, p. 4). The Navy report also concluded that training and manning are two (of 
several) contributors to fleet material readiness and that the changes in each resulted to 
negative impacts in material readiness (GAO, 2011). 

As a result, the Navy announced plans to change aspects of their training and 
readiness strategies. GAO found, however, that the Navy “did not perfonn sufficient 
analysis to verify the infonnation used to support its conclusions and recommendations” 
(GAO, 2011, p. 6). The Navy used information from old reports and inspections, but did 
not conduct any independent analysis to check the validity of the data or assumptions 
contained in those reports. While the recommended changes may prove to be effective, 
they are not rooted in the sort of analysis that is required before coming to such decisions. 
The analysis provided in Chapters V-VII of this thesis will make independent analysis of 
a particular system, the AN/SQQ-89(v) sonar, to show the impacts of the change to CBT 
on maintenance. 

G. SUMMARY 

The Navy’s Revolution in Training strategy attempted to take advantage of e- 
Learning technology to achieve savings in cost and training time while maintaining equal 
or superior quality of learning. Several of the ERNT recommendations were 
implemented, resulting in the consolidation of training commands and the 
implementation of CBT in A and C schools. Navy IG and GAO reports demonstrated that 
while cost and training time savings were achieved, the quality of sailor reporting to the 
Fleet was not as well prepared as ILT-trained sailors of the past. The result is that poorly- 
trained sailors may have contributed to declining material readiness in the Fleet. The next 
section of this thesis examines Navy maintenance practices and highlights the findings of 
the 2010 Fleet Review Panel on Surface Force Readiness report. 
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III. MAINTENANCE 


A. INTRODUCTION 

Navy maintenance occurs on three levels: organizational level (O-level), 
intennediate maintenance (IM) activities, and depot level. This section of the thesis will 
discuss all three maintenance levels. Additionally, this section will discuss changes made 
to the maintenance process in 2003 which were reported on in the 2010 Fleet Review 
Panel on Surface Force Readiness (known as the Balisle Report for its chainnan Vice 
Admiral (VADM, Retired) Phillip Balisle), a report that discussed declining Fleet 
readiness as a result of changes to training, maintenance and manning policies in the 
early 2000s. 

B. SHIPBOARD MAINTENANCE AND ORGANIZATIONAL LEVEL 

MAINTENANCE 

Shipboard maintenance begins with the Planned Maintenance System (PMS). 
PMS is governed by Naval Sea Systems Command (NAVSEA) Instruction 4790.8B, 
Ship’s Maintenance and Material Management (3-M) Manual. The instruction outlines 
the requirements for PMS on shipboard systems and equipment (Naval Sea Systems 
Command, 2003). The purpose of PMS is to provide ships with the means to plan, 
schedule, and perfonn preventive maintenance on ships and to identify potential 
equipment problems before the equipment fails. 

Planned maintenance is scheduled by the Work Center Supervisor (WCS) for all 
equipment in the division and is recorded on the PMS schedule. Maintenance personnel 
must be 3-M qualified and meet rate (skill level) requirements for perfonning a particular 
PMS check as specified on the Maintenance Index Page (MIP) (Naval Sea Systems 
Command, 2003). 

If corrective maintenance is required, the maintenance is reported, scheduled, and 
performed through organizational level (O-level) shipboard maintenance. All 
maintenance actions, including deferred maintenance actions, require a Job Sequence 
Number (JSN). Deferred maintenance actions refer to maintenance that either requires 
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assistance from an outside activity or work center, is not expected to be accomplished by 
ship’s force during the prescribed time period, is an uncorrected discrepancy noted by an 
outside inspecting agency, or is required to correct an existing discrepancy that could 
cause harm to personnel or equipment. Ah deferred maintenance actions are recorded on 
the Current Ship’s Maintenance Project (CSMP), a consolidated list of ah active ship’s 
maintenance actions that provides an accurate picture of the ship’s material condition. 
The CSMP is used by outside activities to schedule repair actions. Ship maintenance 
actions are reported in Navy Visibility and Management of Operating and Support Costs 
(VAMOSC), under Unit Level Consumption and Manhours—Organizational Corrective 
Maintenance. 

C. INTERMEDIATE MAINTENANCE (IM) 

IM is “normally performed by Navy personnel onboard tenders, repair ships, 
Shore Intermediate Maintenance Activities (SIMAs), aircraft carriers, and fleet support 
bases.” (Naval Sea Systems Command, 2003, pp. 1-5) IM jobs are deferred CSMP jobs 
that are beyond the capability of ship’s force and are sent off-ship for completion. IM is 
tracked in Navy VAMOSC under Maintenance - Intermediate. 

D. DEPOT LEVEL MAINTENANCE 

Depot maintenance jobs are deferred CSMP jobs that are beyond ship’s force 
capability and are performed by outside activities such as public shipyards. Depot level 
maintenance “requires major overhaul or a complete rebuilding of parts, assemblies, 
subassemblies, and end items, including the manufacturing of parts, modifications, 
testing, and reclamation.” (Naval Sea Systems Command, 2003, pp. 1-5). Depot 
maintenance is reported in Navy VAMOSC under Maintenance and Modernization - 
Depot, Other Depot. 

E. BALISLE REPORT 

In 2009 VADM (Ret.) Phillip Balisle was directed to conduct a Fleet Review 
Panel (FRP) of surface force material readiness. The review covers the “man, train, and 
equip” domain areas and studies the effects of changes in training, manning, and 
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maintenance in the 2000s and how these changes impacted Fleet material readiness 
(Balisle, 2010). Since training was discussed in depth in Chapter II, this review of the 
Balisle Report will focus on the changes in manning and maintenance. 

1. Optimum Manning 

The concept of “optimum manning” was introduced by the CNO in 2001. The 
purpose of “optimum manning” is to maximize efficiency and ensure that ships have the 
proper amount of personnel on board to support mission accomplishment. Optimum 
manning is based on manning levels required for underway watch standing requirements 
and does not take other duties such as OJT, maintenance, and space preservation, or in¬ 
port work requirements into account. As a result of “optimum manning,” 4,052 billets 
were removed from Navy ships from 2001-2009 (Balisle, 2010). 

While billets were removed from ships, requirements such as maintenance, 
damage control watches, training, and in port duties were not reduced. The result is that 
there are fewer people available to meet the same requirements. This causes major 
problems for ships like the DDG-51 class, where manning was reduced from 317 to 254 
sailors from 1998-2009 (Balisle, 2010). Since the optimum manning initiative relies on 
the right sailors with the right skills, it is highly dependent on the revolution in training. 
The failures of revolution in training described in the previous section exacerbated the 
problems experienced with the optimum manning initiative since sailors were not 
arriving on board with the right KSATs. The result is undennanned ships with poorly 
trained sailors with not enough time or know-how to perfonn routine maintenance 
actions. 

In addition to reduced fleet manning, shore facilities also received manning cuts. 
By 2006 SIMAs and Regional Support Organizations (RSOs) were disestablished 
(Balisle, 2010). This means that maintenance that is intended for IMAs is pushed back to 
ship’s force, which is undennanned and poorly trained. 
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2 . 


Maintenance 


Class Maintenance Plans (CMPs), a maintenance program designed to ensure that 
ships meet or exceed their designated service lives, were discontinued when the 
organization that managed CMPs was disestablished in 1995. In 2000, Integrated Class 
Maintenance Plans (ICMPs) were established to fill the void. ICMPs are managed by a 
port engineer, the chief engineer, and the ship’s commanding officer. The intent of the 
ICMP is that ships would manage their own maintenance plans in conjunction with the 
port engineer, but typically, near-tenn maintenance takes precedence over long-term 
goals (Balisle, 2010). 

As previously noted, closures of SIMAs and the regionalization of maintenance 
activities resulted in backlogs in maintenance. In addition to the shrinking shore 
workforce, maintenance availabilities were shortened from 15 weeks to 9 weeks (Balisle, 
2010). These actions result in equipment being out of commission for longer periods of 
time. 

Finally, changes in PMS were made because ships couldn’t meet maintenance 
requirements due to reduced manning. Maintenance requirements were either eliminated 
or extended in periodicity. The intent was to shift maintenance requirements to shore 
facilities, but since manning was reduced ashore, many requirements went away 
completely (Balisle, 2010). The elimination and extension of maintenance requirements 
can lead to more opportunities for equipment to become inoperable, resulting in degraded 
Fleet readiness. 

F. SUMMARY 

As designed, PMS is intended to keep equipment operational by conducting 
scheduled maintenance checks and correcting discrepancies as they are found. 
Discrepancies are recorded with a JSN and included on the CSMP. Deferred maintenance 
is sent to IMAs or Depot level repair as required. The Balisle Report, however, found that 
changes in manning, training, and maintenance put severe stresses on the system resulting 
in degraded Fleet material readiness. Poorly trained sailors reporting to “optimum 
manned” ships were unable to keep up with maintenance requirements. At the same time, 
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SIMAs were closing and maintenance availabilities were shortened. Backlogged jobs 
resulted in increased maintenance costs because more time was required to complete the 
work. The Balisle report showed that changes in the training program were related to 
maintenance and material readiness. The sections that follow will attempt to show a 
direct relationship between training and maintenance as it relates to the AN/SQQ-89(v) 
sonar system. 


33 



THIS PAGE INTENTIONALLY LEFT BLANK 


34 



IV. DEPARTMENT OF THE NAVY BUDGET 


Training elements of DON Budget Reports from FYs 2000 to 2012 were analyzed 
to detennine the amount of money spent on training each year during the 2000s. The 
ERNT Revolution in Training report and subsequent Navy IG and GAO reports cited 
reduced training costs by relating training as a percentage of the overall budget. In order 
to further examine this issue, data from DON Biennial Budget Estimates for Operations 
and Maintenance (O&M) reports from FYs 2000 to 2012 were used to gather budget 
information for the following categories: 

• Training and Recruiting 

o Recruit Training 
o Specialized Skill Training 

• Ship Operations 

o Ship Operational Support and Training 
o Intennediate Maintenance 
o Depot Level Maintenance 

• Ship Maintenance 

o Emergent Repair 

o Intennediate Maintenance (Note: IM budget items were moved 
from Ship Operations to Ship Maintenance in FY2003) 

All budget reports included 3 years of budget data. For instance, the FY2000 
report included budget data for FYs 1998, 1999, and 2000. The numbers contained in the 
report represent the Total Obligation Authority (TOA) for the given FY in the last year it 
was reported. 

For example, budget data for FY2000 were gathered from the FY2000 budget 
report. TOA is: 
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The sum of: 1.) all budget authority (BA) granted (or requested) from the 
Congress in a given year, 2.) amounts authorized to be credited to a 
specific fund, 3.) BA transferred from another appropriation, and 4.) 
unobligated balances of BA from previous years which remain available 
for obligation. In practice, this tenn is used primarily in discussing the 
DoD budget, and most often refers to TOA as “direct program” which 
equates to only (1) and (2) above. (Department of Defense, 2011) 

Budget information for the categories listed above was gathered for FYs 2000 

through 2010 and converted to constant FY2010 dollars. The following data were 

gathered (Table 1): 
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*AII figures in constant FY10 dollars 

FY 

Training and 

Recruiting 

($M) 

Recruit 
Training ($T) 

Specialized 
Skill Training 
($T) 

Ship 

Operational 
Support and 
Training ($T) 

Intermediate 

Maintenance 

($T) 

Depot Level 

Maintenance 

($M) 

Emergent 
Repair ($T) 

Total DON 
Budget ($M) 

2000 

2,358 

6,772 

327,042 

684,407 

474,492 

3,277 


112,079 

2001 

2,541 

7,086 

347,367 

635,324 

480,792 

3,153 


118,345 

2002 

2,634 

6,962 

395,360 

597,769 

474,449 

3,533 


127,071 

2003 

2,883 

7,427 

455,041 

705,575 

506,262 

5,473 

362,044 

146,816 

2004 

2,158 

11,850 

489,309 

740,478 

710,710 

4,527 

285,758 

140,378 

2005 

2,219 

7,775 

509,350 

697,400 

611,146 

4,799 

249,499 

149,122 

2006 

2,232 

9,638 

548,727 

671,433 

673,440 

4,625 

228,398 

156,399 

2007 

2,262 

9,941 

575,365 

678,651 

719,422 

4,369 

225,695 

159,355 

2008 

2,314 

10,224 

545,769 

718,552 

727,442 

5,127 

286,311 

166,884 

2009 

2,296 

11,357 

602,726 

697,680 

845,443 

5,215 

242,450 

167,982 

2010 

3,204 

10,437 

593,979 

717,586 

1,067,823 

6,265 

303,110 

176,979 


Table 1. DON Budget Data FYs 2000-2010 (constant 2010 dollars) (Department of the Navy, 2000—2010) 
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Figure 1 shows that from FYs 2000-2010, the Training and Recruiting budget was 
fairly stable. The budget rose steadily from FY 2000 to FY2003, but dropped and 
remained stable after that. The growth rate from FYs 2003-2010 was 1.39%. The budget 
change coincides with the training realignment measures in 2003. When examining the 
Specialized Skills Training portion of the training budget, however, training costs nearly 
doubled over the FY 2000-2010 period (Figure 2). The growth rate from FYs 2003-2010 
was 3.82%. This suggests that cost savings were realized in areas other than actual 
training delivery in A and C schools. This could be attributed to realignment of training 
organizations and consolidation of schoolhouses. 



Figure 1. Training and Recruiting ($Millions) FYs 2000-2010 
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Over the same time period, the overall DON budget rose significantly, with a 
growth rate of 2.57% from FYs 2000-2010 (Figure 3). The IG and GAO reports noted 
that training costs had decreased. When examined as a percentage of the total budget, that 
is the case, but only because the total budget increased while the training budget 
remained fairly flat. 
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While the overall training budget was fairly flat during the 2000s, the 
maintenance budget saw increases (Figures 19 and 20). IM budget figures more than 
doubled and Depot level maintenance figures nearly doubled during the 2000s. 



Figure 4. Intermediate Maintenance ($Thousands) FYs 2000-2010 



The rise in IM and depot-level maintenance costs while training costs remained 


flat suggests that there may be a relationship between training policy changes that lead to 
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poorly trained sailors and rising Fleet maintenance costs. Since there are several 
confounding variables at the macro level such as changes in OPTEMPO, manning 
policies, and maintenance policies, it is difficult to draw any conclusions. The aim of this 
thesis is to examine the relationship between training costs and rising maintenance costs 
by focusing on the AN/SQQ-89(v) sonar system. 
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V. AN/SQQ-89(V) SONAR SYSTEM 


A. INTRODUCTION 

The Navy introduced several major changes to training, maintenance and 
manning policies during the early part of the 2000-2010 decade. Interviews conducted in 
2008 with Fleet leadership indicate that senior leaders were unhappy with the quality of 
sailor arriving to the Fleet. Some senior enlisted leaders reported that they had to directly 
supervise or even perform maintenance that was once routine for junior sailors (Naval 
Inspector General, 2009). These interviews suggest that changes in the training program 
may have led to second-order effects in maintenance, causing Fleet maintenance costs to 
rise. On the other hand, the changes made in maintenance and manning may have also 
been contributors. In order to isolate the training variable as the contributor to rising 
maintenance costs, this thesis will focus on the O&S costs of a single Navy system, the 
AN/SQQ-89(v) sonar system, and examine how the conversion to CBT affected 
maintenance costs in that system. Data presented in Chapter VI, Results, will show that 
manning levels for the STG rate did not change significantly from FYs 2000-2010, 
effectively eliminating manning as a contributor for the AN/SQQ-89 O&S costs and 
focusing the study on training and maintenance. 

B. DESCRIPTION 

The AN/SQQ-89(v) surface ship Anti-Submarine (ASW) Warfare combat system 
(referred to as “the 89” in the rest of this paper) is an integrated network of sonar systems 
designed to search, detect, classify, and engage ASW threats. The system is currently 
installed on CG-47 class cruisers, DDG-51 class destroyers, and FFG-7 class frigates. 
The 89 uses a variety of sensors that can transmit (active) and receive (passive) acoustic 
data in order to detect and classify threats. Data from the sensors can be correlated and 
targets can be localized using Target Motion Analysis (TMA) to generate a firing 
solution for weapons systems (Jane's Infonnation Group, 2010). The 89 system must 
include at least two of the following subsystems (Naval Sea Systems Command 
(NAVSEA), 1998): 
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• AN/SQR-19(v) (series) Sonar Receiving Set 

• AN/SQQ-28(v) Sonar Signal Processing System 

• AN/SQS-56 or AN/SQS-53 (series) Sonar Detecting - Ranging Set 

• AN/UYQ-25 (series) Sonar In-Situ Mode Assessment System (SIMAS) 

• MK116 Underwater Fire Control System (UFCS) or Weapons Alternate 
Processor (WAP) (Jane's Information Group, 2010) 

Data from these systems are displayed on operator consoles in the Sonar Control 
room of a surface ship. Operators are designated a particular sensor to monitor (19 
operator, 28 operator, 53 operator, etc.) and the Sonar Supervisor uses the data to 
detennine the classification of a contact. The 89 system can consist of the following 
components: 

1. AN/SQR-19(v) 

The AN/SQR-19, 19A and 19B Sonar Receiving Sets are versions of the Tactical 
Towed Array Sonar (TACTAS). The TACTAS consists of an array of hydrophones 
approximately a mile long towed behind a ship. The hydrophones passively detect sonar 
contacts at long ranges (Jane's Infonnation Group, 2010). The data from TACTAS are 
displayed in Sonar Control and can be used to determine placement of sonobuoys for 
target localization. TACTAS is not available on Flight IIA DDG-51 class destroyers. 

2. AN/SQQ-28(v) 

The AN/SQQ-28 Sonar Signal Processing System is used to process active and 
passive acoustic data from sonobuoys dropped by the SH-60 Seahawk Light Airborne 
Multi-Purpose System (LAMPS) MK III helicopter. The sonobuoys provide long-range 
detection by using helicopters to extend the ship’s nonnal detection range. The 28 is 
available on CG-47 class ships beginning with hull number 56 and in all DDG-51 class 
ships (Jane's Information Group, 2010). 
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3. AN/SQS-56 

The AN/SQS-56 is a hull-mounted active/passive sonar. The 56 is available only 
on FFG 7 class ships. In active mode, the 56 can detect contacts up to 5 nautical miles 
(NM). In passive mode, the 56 can detect to approximately 9 NM (AN/SQS-56 Sonar). 

4. AN/SQS-53 

The AN/SQS-53A, 53B, 53C, and 53D sonars are long range, high power, hull- 
mounted sonars. The 53 series can be found on CG-47 and DDG-51 class ships. The 
sonar has both active and passive modes. Target data from the 53 can be sent to the 
MK116 UFCS to prosecute contacts with shipboard weapons (Jane’s Information Group, 
2010 ). 


5. AN/UYQ-25 

The AN/UYQ-25 SIMAS perfonns environmental assessments to aid Sonar 
Supervisors in deciding the optimal employment of 89 sensors. SIMAS processes 
environmental data from Expendable Bathythermograph (XBT) probes that measure 
temperature versus depth in an operating area. The XBT data are used to determine 
detection range predictions so supervisors can employ their sensors in the best manner 
(Jane's Infonnation Group, 2010). SIMAS is installed on FFG-7, CG-47, and DDG-51 
class ships. 


6. MK116 

The MK116 UFCS takes sensor data from the 19, 28, and 53 and integrates them 
with non-acoustic sensors such as radar and off-ship data from helicopters to generate a 
firing solution using TMA. The firing solution is used to fire shipboard torpedoes or 
Vertical Launch Anti-Submarine Rockets (ASROC). The MK116 is installed on CG-47 
and DDG-51 class ships (Jane's Information Group, 2010). 

7. Weapons Alternate Processor 

The WAP is the fire control processor for FFG 7 class ships. It is integrated with 
the 56 sonar (Jane's Information Group, 2010). 
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C. VARIANTS 

The 89 system consists of 15 different variants. Variants differ based on the 
sensors chosen (19, 28, etc.) and the version of each sensor (53A, 53B, etc.). In this 
report, only variants 2, 3 4, 6, 7 and 9 were studied. These variants were chosen because 
the variants were on board ships prior to the introduction of CBT into the sonar training 
pipeline (2003) and remained on board after CBT was introduced. This allows for 
analysis of ship maintenance trends both prior to and after the introduction of CBT. A 
description of each variant follows and a list of ships per variant is given (Table 2). 

1. AN/SQQ-89(v)2 

For CG-47 class ships, (v)2 consists of the AN/SQR-19(v)l, AN/SQQ-28(v)3, 
AN/SQS-53B, AN/UYQ-25, and MK116. 

For FFG-7 class ships, (v)2 consists of the AN/SQR-19(v)2, AN/SQQ-28(v)2, 
AN/SQS-56, AN/UYQ-25A, and WAP (Naval Sea Systems Command (NAVSEA), 
1998, pp. G-5). 

2. AN/SQQ-89(v)3 

The AN/SQQ-89(v)3 is available only on CG-47 class ships. It consists of the 
AN/SQR-19(v)l, AN/SQQ-28(v)3, AN/SQS-53B(v)2, AN/UYQ-25, and MK116 Mod 6 
(Naval Sea Systems Command (NAVSEA), 1998, pp. G-5). 

3. AN/SQQ-89(v)4 

The AN/SQQ-89(v)4 is available only on DDG-51 class ships. It consists of the 
AN/SQR-19A(v)4, AN/SQQ-28(v)3, AN/SQS-53C(v)l, AN/UYQ-25A(v)2, and MK116 
Mod 7 (Naval Sea Systems Command (NAVSEA), 1998, pp. G-5). 

4. AN/SQQ-89(v)6 

The (v)6 is available on CG-47 class and DDG-51 class ships. For CG-47 class 
ships, (v)6 consists of the AN/SQR-19B(v)l, AN/SQQ-28(v)9, AN/SQS-53C(v)l, 
AN/UYQ-25A(v)2, and MK116 Mod 7 (Naval Sea Systems Command (NAVSEA), 
1998, pp. G-5). 
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5. AN/SQQ-89(v)7 

The (v)7 is available on CG-47 class ships and consists of the AN/SQR-19B(v)l, 
AN/SQQ-28(v)9, AN/SQS-53B(v)2, AN/UYQ-25B(v)2, and MK116 Mod 7 (Naval Sea 
Systems Command (NAVSEA), 1998, pp. G-5). 

6. AN/SQQ-89(v)9 

The (v)9 is available on FFG-7 class ships and consists of the AN/SQR-19B(v)2, 
AN/SQQ-28(v)9, AN/SQS-56, AN/UYQ-25A(v)2, and WAP (Naval Sea Systems 
Command (NAVSEA), 1998, pp. G-5). 


47 



(V)2 

SHIP 

HOMEPORT 


(V)3 

SHIP 

HOMEPORT 


(V)6 

SHIP 

HOMEPORT 

CG 55 

LEYTE GULF 

Norfolk, VA 


CG 56 

SAN JACINTO 

Norfolk, VA 


CG 68 

ANZIO 

Norfolk, VA 

FFG 8 

MCINERNEY 

Mayport, FL 


CG 57 

LAKE CHAMPLAIN 

San Diego, CA 


CG 69 

VICKSBURG 

Mayport, FL 

FFG 28 

BOONE 

Mayport, FL 


CG 58 

PHILIPPINE SEA 

Mayport, FL 


CG 70 

LAKE ERIE 

Pearl Harbor, HI 

FFG 29 

STEPHEN W GROVES 

Mayport, FL 






CG 71 

CAPE ST GEORGE 

San Diego, CA 

FFG 32 

JOHN HALL 

Mayport, FL 


(V)4 

SHIP 

HOMEPORT 


CG 72 

VELLA GULF 

Norfolk, VA 

FFG 33 

JARRET 

San Diego, CA 


DDG51 

ARLEIGH BURKE 

Norfolk, VA 


DDG 52 

BARRY 

Norfolk, VA 

FFG 36 

UNDERWOOD 

Mayport, FL 






DDG 53 

JOHN PAUL JONES 

San Diego, CA 

FFG 38 

CURTS 

San Diego, CA 






DDG 54 

CURTIS WILBUR 

Yokosuka, Japan 

FFG 39 

DOYLE 

Mayport, FL 






DDG 55 

STOUT 

Norfolk, VA 

FFG 40 

HALYBURTON 

Mayport, FL 






DDG 56 

JOHN S. MCCAIN 

Yokosuka, Japan 

FFG 41 

MCCLUSKY 

San Diego, CA 






DDG 57 

MITSCHER 

Norfolk, VA 

FFG 42 

KLAKRING 

Mayport, FL 






DDG 58 

LABOON 

Norfolk, VA 

FFG 43 

THACH 

San Diego, CA 






DDG 59 

RUSSELL 

Pearl Harbor, HI 

FFG 45 

DE WERT 

Mayport, FL 






DDG 60 

PAUL HAMILTON 

Pearl Harbor, HI 

FFG 46 

RENTZ 

San Diego, CA 






DDG 61 

RAMAGE 

Norfolk, VA 

FFG 47 

NICHOLAS 

Norfolk, VA 






DDG 63 

STETHEM 

Yokosuka, Japan 

FFG 48 

VANDEGRIFT 

San Diego, CA 


(V)7 

SHIP 

HOMEPORT 


DDG 64 

CARNEY 

Mayport, FL 

FFG 49 

ROBERT G BRADLEY 

Mayport, FL 


CG 66 

HUE CITY 

Mayport, FL 


DDG 65 

BENFOLD 

San Diego, CA 

FFG 53 

HAWES 

Norfolk, VA 


CG 67 

SHILOH 

Yokosuka, Japan 


DDG 66 

GONZALEZ 

Norfolk, VA 

FFG 55 

ELROD 

Norfolk, VA 






DDG 67 

COLE 

Norfolk, VA 

FFG 56 

SIMPSON 

Mayport, FL 


(V)9 

SHIP 

HOMEPORT 


DDG 68 

THE SULLIVANS 

San Diego, CA 

FFG 57 

REUBEN JAMES 

Pearl Harbor, HI 


FFG 37 

CROMMELIN 

Pearl Harbor, HI 


DDG 69 

MILIUS 

San Diego, CA 

FFG 58 

SAMUEL B ROBERTS 

Mayport, FL 


FFG 50 

TAYLOR 

Mayport, FL 


DDG 70 

HOPPER 

Pearl Harbor, HI 

FFG 59 

KAUFFMAN 

Norfolk, VA 


FFG 51 

GARY 

San Diego, CA 


DDG 71 

ROSS 

Norfolk, VA 

FFG 60 

RODNEY M. DAVIS 

Everett, WA 


FFG 52 

CARR 

Norfolk, VA 


DDG 72 

MAHAN 

Norfolk, VA 

FFG 61 

INGRAHAM 

Everett, WA 


FFG 54 

FORD 

Everett, WA 


DDG 73 

DECATUR 

San Diego, CA 






DDG 74 

MCFAUL 

Norfolk, VA 








DDG 75 

DONALD COOK 

Norfolk, VA 








DDG 76 

HIGGINS 

San Diego, CA 








DDG 77 

O'KANE 

Pearl Harbor, HI 








DDG 78 

PORTER 

Norfolk, VA 


Table 2. List of Ships and AN/SQQ-89(v) System Variants used in this study 
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D. TRAINING 

All STGs attend STG A school. At A school, students learn the basic principles of 
the STG rating including oceanography and principles of sound. Following A school, 
STGs are sent to different courses depending on whether they are operators or 
operator/maintainers. STGs that are strictly operators are sent to a Sonar Operator course, 
where they learn how to operate the specific 89 variant of the ship to which they will be 
sent. Maintainers are sent to C school, where they learn the technical skills required to 
maintain the equipment they will work on upon reporting to their ship. Operators and 
maintainers are assigned NECs based on the schools they attend (Navy Personnel 
Command, 2012). For example, an STG could be designated with the 0455 NEC, 
AN/SQQ-89(v)4/6 Active Sonar Level II Technician with the following job description: 

Performs organizational level maintenance on subsystems including; 
AN/SQS-53C Sonar System, AN/UQN-4/4A Sonar Sounding Set and the 
AN/WQC-2/2A Sonar Communications Set. Identify acoustic signatures 
to source. Operate the following subsystems; AN/SQQ-28(V) Sonar 
Signal Processing System, AN/SQS-19(V) TACTAS, AN/SQS-35C(V) 

1/2/3 and AN/UYQ-25A(V)2 or AN/UYQ-25B SIMAS (Navy Personnel 
Command, 2012, p. 16 (Chapter 4)). 

Data were not available to show how STG course lengths were affected by the 
conversion to CBT. The 2009 Navy IG report, Computer Based Training, examined the 
course lengths of 22 A and C schools for ILT and CBT. On average, CBT course lengths 
were 26% shorter than ILT course lengths (Naval Inspector General, 2009). This thesis 
will assume a 26% course length reduction for STG schools. 

E. SUMMARY 

The 89 sonar system is an integrated set of sonar tools used in ASW. This study 
will examine the O&S costs of the 89 system and determine whether there is a 
relationship between the conversion to CBT in the STG rating and increased maintenance 
costs for the 89. The chapters that follow examine the methodology for examining the 
training and maintenance relationship and report the findings of the study. 
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VI. METHODOLOGY 


A. INTRODUCTION 

The final question posed by this thesis was, “if sailors trained with CBT had 
lower knowledge and skill levels, did this contribute to increased O&M costs?” 
Attempting to answer this question at the macro level proved to be difficult because there 
appeared to be several variables that could impact Fleet maintenance costs such as 
OPTEMPO, Fleet manning, GWOT, maintenance policy changes, and the conversion to 
CBT. Data measuring training effectiveness were not available either pre- or post-CBT 
because there were no measurement standards in place. Navy budget data and O&S data 
were available for analysis, but the data available do not present an obvious answer; data 
had to be combined from several sources and, in some cases, assumptions and estimates 
of data had to be made in order to reach a conclusion. The sections that follow will 
describe the data sources used for the study of the AN/SQQ-89(v) sonar system and the 
methodology employed to organize, filter and clean the data (data normalization) for 
further analysis. 

B. DATA SOURCES AND ORGANIZATION 

Data were collected from several DoD sources in order to determine the impact of 
the CBT conversion in the STG rating on maintenance in the fleet. PEO IWS5 provided a 
list of ships equipped with the AN/SQQ-89(v) sonar system. Naval Personnel Command 
(NPC) provided STG manning data for CG-47, DDG-51, and FFG-7 class ships. Navy 
Visibility and Management of Operating and Support Costs (VAMOSC) provided O&S 
cost data, underway steaming days, and selected non-cost data for ships equipped with 
the AN/SQQ-89 sonar system. 

CBT was introduced full-time into the training pipeline in FY2003, after the 
recommendations of the ERNT report (Naval Inspector General, 2009). This study 
focuses on FYs 1999 through 2010 to capture data prior to and after the introduction of 
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CBT. Initially, FYs 1995 through 1998 were also considered, but there were not enough 
data available during this timeframe for most data categories. The raw data provided were 
analyzed to reveal relationships between selected data sets. 

1. Order of Battle (OOB) 

PEO IWS 5 provided a list of ships equipped with the 89 system. The list included 
ship class, ship name, hull number, homeport, and 89 variant number. Only ships with 
AN/SQQ-89(v) variants on board both prior to and after introduction of CBT into the 
STG rate training pipeline were considered. The initial list provided by PEO IWS5 
included all ships of the CG-47, DD-963, DDG-51, and FFG-7 classes. To narrow the 
ship list to match the scope of our study, ships were removed from the data set if: 

• The ship was decommissioned during the FY95-06 timeframe 

• The ship received a variant upgrade 

• The ship was commissioned FY2000 or later 

• The ship was outfitted with a variant introduced after FY03 

Using the above criteria, the ship list was reduced to 68 ships. All Spruance (DD- 
963) class ships were removed due to decommissioning. Only ships with (v)2, 3, 4, 6, 7, 
or 9 were considered since they existed both prior to and after the introduction of CBT. 
Additionally, several ships of the FFG-7 and CG-47 classes decommissioned and several 
CG-47 class ships received variant upgrades. USS ANTIETAM (CG 54) was eliminated 
because data reporting stopped in 2004. The final list of 68 ships was separated by variant 
and includes homeport (Table 2, Chapter V.C). 
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2. NPC Manning Data 

STG manning data were analyzed to detennine whether STG manning levels were 
an important variable in the study since the Balisle report included the shift to optimal 
manning as a contributor to degraded Fleet readiness. Manning data provided by NPC 
initially included manning numbers for all ratings on board the ships of the CG 47, DDG 
51, and FFG 7 classes. The following data points were given for each fiscal year for all 
ships of the CG-46, DDG-51, and FFG-7 classes (sample provided in Table 3): 

• Billets Authorized (BA) for pay grades E1-E3, E4, E5, E6, E7, E8 and E9 

• Navy Manning Plan (NMP) for pay grades E1-E3, E4, E5, E6, E7, E8 and 
E9 

• Current On Board (COB) for pay grades E1-E3, E4, E5, E6, E7, E8 and 
E9 

BA refers to “a funded military manpower space, which has been authorized by 
CNO (Department of the Navy, 2003, p. 1).” NMPs are used by NPC to detennine the 
‘fair share’ of personnel from a particular rating which are distributed from the personnel 
inventory to a particular ship (Department of Defense Inspector General, 2004). COB 
refers to the number of personnel on board a given ship in a particular rating. 
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(v)3 

UIC 

HULL NO. 

SHIP NAME 

RATE 

Billets 

Authorized 

(BA) 

BA E1-E3 

BA E4 

BA E5 

BA E6 


21389 

CG 56 

SAN JACINTO 

STG 

16 

1 

6 

5 

3 


21428 

CG 57 

LAKE CHAMPLAIN 

STG 

16 

3 

6 

4 

2 


21429 

CG 58 

PHILIPPINE SEA 

STG 

16 

1 

7 

4 

3 


Table 3. Sample Manning Data - FY99 BA for (v)3 (Navy Personnel Command) 
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The data were filtered to include only the STG rating and the ships selected in 
Table 1. A variant manning average (VMA) was detennined for each variant in a FY to 
show the average manning levels for the STG rating on a particular AN/SQQ-89(v) 
variant in the selected FY. This allowed for trend detection for each variant to note 
whether manning for a particular variant was changing from year-to-year. For variants 
installed on multiple ship classes, separate VMAs were calculated for each class of ship. 
For example, (v)2 was installed on ships of the CG-47 and FFG-7 classes, so VMAs were 
calculated separately for each class (Table 4). 
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(v)6 

UIC 

HULLNO. 

SHIP NAME 

RATE 

Billets 

Authorized 

(BA) BAE1-E3 

BA E4 

BA E5 

BA E6 


21658 

CG 68 

ANZIO 

STG 

16 

1 

7 

4 

3 


21684 

CG 69 

VICKSBURG 

STG 

16 

1 

7 

4 

3 


21827 

CG 70 

LAKE ERIE 

STG 

18 

3 

7 

4 

3 


21828 

CG 71 

CAPE ST GEORGE 

STG 

16 

1 

7 

4 

3 


21829 

CG 72 

VELLA GULF 

STG 

16 

1 

7 

4 

3 

(v)6 CG AVG 



16.4 

1.4 

7 

4 

3 


Table 4. Sample Manning Data with VMA (Full data set not shown) (Navy Personnel Command) 
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3. Visibility and Management of Operating and Support Costs 
(VAMOSC) 

Navy VAMOSC provided a database of O&S costs for the AN/SQQ-89(v) system 
covering FYs 1995 through 2010. The O&S data given were a line-by-line description of 
all costs associated with the 89 system. Non-cost data such as manhours and number of 
corrective maintenance actions were also given. Cost figures were given in then-year and 
constant FY2 Oil dollars. In addition to the overall 89 system data, detailed ship data 
were also given for the selected ships. Detailed ship data did not include training costs, 
which had to be estimated (description of estimation techniques is discussed later in the 
report). O&S costs for individual ships carrying the 89 system were analyzed to 
determine the relationship between training and maintenance. Definitions of key O&S 
terms used in this study are listed in Appendix A. Since this study focuses on training and 
maintenance, not all O&S data points provided by VAMOSC were relevant. Table 5 
shows the O&S data points used for this study. 
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S 


Direct Unit Cost 


Direct Unit Cost 


Direct Unit Cost 


Direct Unit Cost 


Maintenance-Intermediate 


nrimreBffil 


Maintenance-Intermediate 


lMiHaiw 


Maintenance & Modernization 


Maintenance & Modernization 


Maintenance & Modernization 


Maintenance & Modernization 


Other Operating & Support 


Other Operating & Support 


Other Operating & Support 


Unit Level Consumption 


Unit Level Consumption 


Unit Level Consumption 


Unit Level Consumption 


Labor-Intermediate Maintenance 


Labor-Intermediate Maintenance 


Material-Intermediate Maintenance 


Material-Intermediate Maintenance 


Equipment Rework 


Equipment Rework 


Other Depot 


Other Depot 


Training 


Repair Parts & Repairables 


Repair Parts & Repairables 


Repair Parts & Repairables 


Repair Parts & Repairables 


Labor - Afloat Intermediate Maintenance 


Labor - Ashore Intermediate Maintenance 


Material - Afloat Repair Parts Intermediate Maintenance 


Material - Ashore Repair Parts Intermediate Maintenance 


Equipment Rework ■ Contractor ■ Program Office 


Equipment Rework ■ Government ■ Program Office 


Maintenance - Depot - Public Shipyards - Systems 


Maintenance ■ Depot ■ Private Shipyards ■ Systems 


Training-Program Office 


Training-NETPDTC 




Training 


Engineering & Technical Services (ETS) 4,3,3 Engineering & Technical Services (ETS) ■ Program Office 


Number of Ships or Number of Systems 


Personnel Trained D.1 Personnel Trained-Program Office 


Personnel Trained D.2 Personnel Trained-NETPDTC 


Personnel Trained 


Manhours-Org Corrective Maint 


Manhours - Intermediate Maintenance - Afloat 


Manhours - Intermediate Maintenance - Ashore 


Number of Corrective Maintenance Actions - O&l Level 


Manhours - Depot Maintenance - Public 


1.2.2.21Repair Parts -Organizational Repair Parts 


1.2.2.3 


1.2.2.3 


1.2.2.3 


es 

1,2,2,3.1 

Replenishr 

1,2,2,3,1.1 

es 

1,2.2,3.1 

Replenishr 

1.2.2.3.1.2 

es 

1.2.2.3.2 

Exchanges-LOGCEN 



— 



Table 5. O&S Data Points (Navy VAMOSC) 
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All O&S data were analyzed using constant FY11 dollars. All fields were 
summed to yield a total O&S cost for each FY. All data fields relevant to training and 
maintenance study, including non-cost data points such as personnel trained and man¬ 
hours were analyzed further. 

Navy VAMOSC also provided detailed ship O&S data and ship underway days 
for analysis. Underway days were studied to detennine whether OPTEMPO was related 
to number of maintenance actions or maintenance cost. Detailed ship O&S data were 
used to analyze ship-to-ship trends and detennine whether variant, homeport, or ship 
class were significant factors. For each data set, only ships with complete or near- 
complete data were selected. Near-complete is defined as no more than 3 years of 
missing data. Due to inconsistencies in data reporting, some data sets had larger sample 
sizes than others. There were not sufficient data present for any ship to analyze the 
following data sets: 

• Labor - Afloat Intermediate Maintenance Manhours 

• Materials - Afloat Repair Parts Intennediate Maintenance Cost 

• Materials - Ashore Repair Parts Intennediate Maintenance Cost 

• Replenishment Spares - Logistic Center (LOGCEN) Cost 

C. CONSOLIDATED DATA TABLE 

After filtering and normalizing the raw data in OOB, 89 Sonar System data, 
personnel data, and detailed ship O&S Cost, the data were consolidated into a single file 
for further analysis. Detailed ship O&S cost data were used instead of the overall O&S 
cost data for the 89 system because the detailed ship data provided more granularity and 
allowed for trend detection from ship to ship. The consolidated file is listed by fiscal year 
and includes a data separator to separate pre- and post-CBT years. Ships are identified by 
class, hull number and ship name. For each ship, 89 variant, 89 install year, homeport, 
E1-E6 STG manning, underway days, and VAMOSC O&S data are given. 

All maintenance data provided by Navy VAMOSC were given in a detailed ship 
file. Additionally, Navy VAMOSC provided overall AN/SQQ-89 O&S data that 
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represented all U.S. Navy ships that had an AN/SQQ-89 sonar system installed. Since 
ship-specific training and O&S data were not provided, they were estimated and allocated 
to each ship using methodology described in the paragraphs to follow. 

The quantity of 89 systems and ships in operation is given for each fiscal year in 
the O&S data. Using information provided by Program Executive Office Integrated 
warfare System (PEO IWS) 5, the number of ships per class for each fiscal year was 
detennined. Ships are separated by class in order to allocate training and O&S data to 
each ship based on average manning levels for each ship class. 

Manning data provided by NPC were used to estimate the average number of 
STG personnel per ship class. The number of STG personnel for each fiscal year was 
detennined by multiplying the number of ships in a class by the average manning number 
for that class, and summing each ship class manning number together. 

The average cost per STG personnel was determined by dividing Training - 
Naval Education and Training Professional Development and Technology Center 
(NETPDC) and O&S (Total) by the number of personnel for each fiscal year. Training 
and O&S cost for each ship was detennined by multiplying the average cost per STG by 
the average number of STG per ship class. Using this method, Training and O&S costs 
for each ship in a particular class are identical (based on an average). In reality, this is not 
the case, but given the lack of detailed ship infonnation for these data points, this 
estimation method represents a reasonable way to allocate detailed Training and O&S 
data to individual ships. 

Data in the consolidated data matrix were cleaned by removing outliers. 
Approximately 1% of all data points collected were outliers. Outliers were detennined to 
be any data point that exceeded plus or minus three standard deviations from the mean. 
The method for detennining outliers is described in the paragraphs to follow. 

Pivot tables were constructed with Hull Number as the row field and Fiscal Year 
as the column field. Data were populated into this format for manning, underway days, 
and selected VAMOSC data. Outliers were determined by using conditional formatting to 
identify which data points laid plus or minus three standard deviations from the mean. 
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The highlighted data points were removed and the remaining data were placed into a 
“cleaned data” file. The final consolidated data file was used to conduct analysis to 
detennine the relationship between the introduction of CBT and maintenance on the 
AN/SQQ-89(v) system. 

The consolidated data file represents all data deemed relevant to studying the 
relationship between the conversion to CBT and increased maintenance costs in the 89 
sonar system. All data contained in the consolidated file were analyzed to determine an 
answer to the final question posed in the thesis, “if sailors trained with CBT had lower 
knowledge and skill levels, did this contribute to increased O&M costs?” The results of 
this analysis are discussed in the next chapter. 

D. SUMMARY 

Data were collected from several Navy sources in order to answer the final thesis 
question. The data represented both cost and non-cost elements that were deemed 
essential to answering the question. Relevant data was assembled into a consolidated data 
file for analysis. Results of this analysis are given in Chapter VII, Results. 
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VII. RESULTS 


A. ANALYSIS 

The data collected from NPC, PEO IWS5 and VAMOSC were collected into a 
consolidated data file in order to answer the final thesis question, “if sailors trained with 
CBT had lower knowledge and skill levels, did this contribute to increased O&M costs?” 
Data was analyzed using methods including graphical analysis, single-factor regression 
analysis, and paired t-tests to detennine answer to the final thesis question. Further 
explanation of the methods used in this study, and results, are discussed in this chapter. 

1. NPC Manning Data 

For each variant, variant moving averages (VMAs) for FY99 through FY10 were 
tabulated and plotted to determine if there were significant changes in manning from year 
to year. In the Navy maintenance community, pay grades El through E6 perfonn the bulk 
of preventative and corrective maintenance actions. Only pay grades El through E6 were 
studied for manning level changes (Figure 6). In this example, there do not appear to be 
any noticeable manning trends. For (v)2 CGs, Billets Authorized (BA) remained constant 
across all fiscal years, while Navy Manning Plan (NMP) and Current On Board (COB) 
had variability. COB is the most telling of the three measures of manning because it 
shows the actual number of STG personnel on board vice a prescribed amount. In this 
case, manning was highly variable from year to year and does not appear to be tied to the 
reduced (optimal) manning concept. The variability in manning for COB may be 
attributed to the early departure of sailors for disciplinary reasons or the acquisition of 
additional sailors due to cross-rating from one job to another. In Figure 6, for example, 
the VMAs in FYs 2003 and 2004 show a great deal of variance which may be attributed 
to the previously noted personnel issues. The example given in Figure 6 is consistent 
through all variants of the 89 system; that is, all variants of the 89 system show stable BA 
manning numbers and variable COB numbers. There does not appear to be a trend 
towards reduced manning in the STG rate from a variant standpoint. 
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Fiscal year 


BA E1-E6 
NMP E1-E6 
COB E1-E6 


Figure 6. (v)2 STG Manning FYs 1999-2010 
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VMAs for ship class (CG-47, DDG-51, FFG-7), regardless of variant, were also 
plotted to detennine whether there were any significant changes in manning for each 
class of ship for FY99 through FY10 (Figure 7). As with VMA’s per variant, the VMA 
per ship class was fairly stable when observing BA and more variable with NMP and 
COB. Again, the VMA for COB was highly variable, but it tended to be higher than the 
VMA for BA and it does not appear that the STG rating was significantly affected by the 
optimal manning concept. The example provided in Figure 7 is indicative of the trends 
exhibited by all ship classes studied; that is, DDG-51 and FFG-7 class ships also showed 
relatively stable BA manning numbers and variable COB numbers. There was no 
noticeable trend towards reduced manning in the STG rating from a ship class 
perspective. 
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•BAE1-E6 
NMP E1-E6 
«COB E1-E6 


Figure 7. CG Class STG Manning, FYs 1999-2010 
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As Figures 6 and 7 indicate, whether analyzing the data from the variant 
perspective or from the ship class perspective, manning may not have a significant impact 
on the study. Across all variants, BA remains relatively consistent through the FYs. In all 
cases, the VMA for BA changed by less than 1, with the exception of FFG manning, 
which decreased by 2 in FY2009. NMP and BA were more variable, but the variability 
was observed from year-to-year and it does not appear that optimal manning in the Fleet 
impacted the STG rating as much as it may have affected other ratings. For this reason, 
manning was not considered to be a significant factor in drawing a conclusion to the 
proposed thesis question. 

2. VAMOSC 

For the overall 89 system O&S components selected for this study, calculations 
were made to detennine what percentage each component was of total O&S costs. This 
was done to detennine spending changes for a particular component from year-to-year 
(Table 6). 
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1995 

1996 

1997 

Unit Level Consumption as % of O&S 

7 . 57 % 

5 . 21 % 

15 . 68 % 

Intermediate Maintenance as%of O&S 

r 

0 . 08 % 

r 

0 . 03 % 

r 

0 . 34 % 

Equipment RE-work as % of O&S 

r 

1 . 05 % 

r 

0 . 06 % 

r 

1 . 31 % 

Depot Maintenance as % of O&S 

r 

0 . 00 % 

r 

0 . 07 % 

r 

0 . 00 % 

Training (Program Office) as % of O&S 

0 . 00 % 

0 . 00 % 

0 . 00 % 

Training (NETPDTC) as % of Program Office 

12 . 40 % 

13 . 55 % 

45 . 54 % 

Training (Total) as % of O&S 

r 

12 . 40 % 

r 

13 . 55 % 

r 

45 . 54 % 

Engineering and Technical Support as % of O&S 

20 . 69 % 

4 . 95 % 

8 . 24 % 

Software Support as % of O&S 

10 . 92 % 

1 . 46 % 

4 . 52 % 

Training Cost (Total) 

r $ 

8 , 343,490 

r $ 

10 , 704,760 

F $ 

19 , 250,317 

Unit Level Consumption Cost (Total) 

$ 

5 , 094,929 

$ 

4 , 118,052 

$ 

6 , 628,471 

Intermediate Maintenance Cost (Total) 

r $ 

50,565 

r $ 

20,681 

r $ 

143,961 

Equipment Rework Cost (Total) 

r $ 

704,760 

$ 

46,567 

F $ 

555,816 

Depot Maintenance Cost (Total) 

r $ 

- 

$ 

52,075 

F $ 

- 


Table 6. Sample O&S Calculations 
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After components were analyzed as a percentage of overall cost, each component 
was analyzed for changes in dollars spent. This was done because it is possible that 
though the percentage in relationship to the overall whole may have changed, the actual 
dollar amount spent on a particular component didn’t change at all due to spending 
changes in other areas. If the dollar amount is relatively unchanged in the case of training 
costs, it would suggest that the claims of cost savings were inaccurate. To illustrate this, 
Figures 8 and 9 show the relationships for Training (Total) and IM. 
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Training (Total) as % of O&S 



Fiscal Year 


Figure 8. Training (Total) as Percentage of O&S Costs 
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Figure 9. Training Cost 
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Figures 8 and 9 show that over the same time period, training as a percentage of 
total O&S costs behaved similarly, but not identically, to overall training costs. 
Generally, it appears that training costs were lower for the 89 system after the 
introduction of CBT, which suggests that the claims of cost savings when converting to 
CBT were accurate in terms of the 89 system. 
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Figure 10. Intermediate Maintenance Cost 
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Figure 10 shows that IM costs increased substantially after the introduction of 
CBT (2003). While IM costs were trending upward prior to 2003, the rapid rise in costs 
after 2003 suggests that there is cause to further analyze the relationship between 
decreasing training costs and increased maintenance costs. 

Non-cost data were also provided and included number of personnel trained, 
maintenance manhours, and number of maintenance actions. The data were used to 
calculate average time (in manhours) spent per maintenance action, cost to train (per 
person) using Program Office dollars, and cost to train (per person) using NETPDTC 
dollars. These numbers were calculated to determine training and maintenance trends pre- 
and post-CBT. For instance, if the average time spent per maintenance action increased, it 
could suggest a backlog of maintenance or a lack of technical competence in performing 
a maintenance action. Since number of personnel trained for the Program Office and 
NETPDTC were unavailable for some FYs, the cost to train data were not useful. 
Manhours spent per maintenance action, however, were useful (Figure 11). Prior to CBT, 
manhours spent per maintenance action were trending upward; after the introduction of 
CBT, manhours per maintenance action remained relatively flat. This suggests that 
manhours per maintenance action may have reacted positively to the conversion to CBT; 
however, this assumes that the types of maintenance actions perform remained relatively 
constant. 
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Average Time (manhours) per 
maintenance action 



Figure 11. Man-hours Spent per Maintenance Action 
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Single-factor regression analysis was performed to explore the relationship 
between total training cost and the O&S variables provided by Navy VAMOSC. Unit 
Level Consumption, IM, Equipment Rework, and Depot Maintenance were selected as 
variables to determine whether changes in training costs resulted in increased 
maintenance costs. The selected variables represent organizational, intermediate, and 
depot level maintenance. Unit Level Consumption is a summation of Organizational 
Repair Parts, Replenishment Spares, and Logistics Center (LOGCEN) exchanges. 
Equipment Rework is a summation of contractor and government Program Office rework 
costs. IM is a summation of afloat and ashore IM labor costs and ashore IM materials 
costs. Depot Maintenance is a summation of private and public shipyard depot costs. 
Training (Total) is a summation of Program Office and NETPDTC training costs. 



R 2 

Unit Level Consumption 

0.033 

Equipment Rework 

0.002 

Intermediate 

Maintenance 

0.348 

Depot Level Maintenance 

0.208 


Table 7. Regression Analysis - O&S Components 
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2 

R measures the percentage of variation in a response variable that is explained by 
the interaction with a predictor variable. An R" value greater than .70 would indicate a 
significant relationship between the factors. In this case, regression analysis shows that 
none of the factors selected show a statistically significant relationship to total training 
cost. The scatter plots shown in Figures 12-15 illustrate the general relationship between 
each of the maintenance categories and training costs. For each maintenance category the 
best-fit line (based on ordinary least squares regression) is drawn and the equation of the 
line, as well as the R“ is shown. The closer the points on the plot are to the fitted line, the 
higher the correlation between the two variables. If the points are scattered all over the 
graph with no apparent pattern, then no relationship is suggested by the plot and further 
analysis will likely prove fruitless. The relationships may be positive or negative 
depending on the slope of the line. A slope of zero indicates no relationship. The first 
two plots (Figures 12 & 13) suggest that there is no relationship between training costs 
and unit level consumption or equipment rework. The Intermediate Maintenance (IM) 
scatter plot (Figure 14) suggests that there may be a relationship between training costs 
and intermediate maintenance and warrants further investigation. Specifically the plot 
suggests that as training costs increase, intermediate maintenance costs decrease. This 
would support the hypothesis that when less money is spent on training (as a result of 
switching to CBT) the maintenance costs will increase. The final plot (Figure 15) appears 
to have an outlier that skews the relationship. Without this data point, there does not 
appear to be a strong relationship between depot maintenance and training costs. Overall, 
the regression analysis suggests that if maintenance variables have a correlated 
relationship with training in the STG rating, there are other factors not identified in this 
study that are having an effect. 
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Figure 13. Training vs. Equipment Rework Scatter Plot 
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Figure 15. Training vs. Depot Maintenance Scatter Plot 

Graphical analysis of several data categories indicated noticeable changes after 
the introduction of CBT. For example, Labor Ashore—Intennediate Maintenance 
Manhours showed significant change (Figure 16). The figure shows number of manhours 
spent on IM for selected ships from FYs 1995 through 2010. Beginning in FY04, the IM 
manhours increased significantly for the selected DDG-51 and CG-47 class ships. While 
this may be partially explained by changes to Navy maintenance policy described in 
Chapter III, it corroborates the evidence suggested in Figure 14, namely that as CBT is 
introduced and training costs decrease, intermediate maintenance hours and cost increase. 
This trend was not as evident, however, for the FFG-7 class. This may be explained by 
funding of the ship class, since many of these ships belong to the Naval Reserve Force 
and it is likely that their funding levels did not change throughout the period studied. 
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Figure 16. Labor Ashore - Intermediate Maintenance Manhours 
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3. 


Consolidated Data 


The cleaned consolidated data were imported into Minitab® (a statistical software 
package) for analysis. Paired t-tests were conducted to determine whether the means of 
paired observations (selected ships pre- and post-CBT) were different. The null hypothesis 
states that the population means are equal while the alternative hypothesis states that the 
means are not equal. A p-value less than the stated level of significance, 0.05 in this case, 
indicates that the means are not equal and the null hypothesis can be rejected. A t-statistic 
less than 0.00 indicates that the first mean sampled is smaller than the second, while a 
positive t-statistic suggests the opposite. In this case, the response variables were tested to 
determine whether there was a significant difference between means pre- (mean 1) and post- 
CBT (mean 2). The null hypothesis is that there is no significant difference in the means 
before and after the introduction of CBT. The alternate hypothesis is that there is a significant 
difference between the means due to CBT. A negative t-statistic indicates that the pre-CBT 
mean was smaller, while a positive t-statistic indicates that the post-CBT mean was smaller. 
Response variables used in this study were: Organizational Parts Cost, Exchanges LOGCEN 
Cost, Manhours Organizational Labor, Training, NETPDTC, O&S Cost, O&S Cost less 
Training, O&S Cost less Training less Maintenance and Modernization, and O&S less 
Maintenance and Modernization (Table 8). 


Response Variable 

Degrees 

of 

Freedom 

T-statistic 

p-value 

Org. Parts Cost 

67 

-3.96 

0.000 

Exchanges LOGCEN Cost 

47 

-8.69 

0.000 

Manhours Org. Labor 

67 

-5.6 

0.000 

Training NETPDTC 

67 

27.06 

0.000 

O&S Cost 

67 

32.12 

0.000 

O&S Cost - Training 

67 

31.44 

0.000 

O&S - Training - Maint. & Mod 

67 

32.38 

0.000 

O&S - Maint. & Mod. 

67 

32.31 

0.000 


Table 8. Paired T-test results 

In all cases, the p-values were less than the level of significance, indicating that 
the null hypothesis should be rejected and the alternate hypothesis should be accepted. 
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This means that there was a significant difference between the means pre- and post-CBT. 
The t-statistics show that the means for Organizational Parts Cost, Exchanges LOGCEN 
Cost, and Manhours Organizational Labor were higher after the introduction of CBT. 
Additionally, the mean costs for Training—NETPDTC, O&S Cost, O&S Cost less 
Training, O&S Costs less Maintenance and Modernization, and O&S Costs less Training less 
Maintenance and Modernization were lower after the introduction of CBT. 

B. SUMMARY 

Manning for the STG rating did not have a significant impact on the results of this 
thesis study. The optimum manning initiative affected some ratings more than others. 
Based on the data presented, the STG rating did not see significant reductions. This does 
not mean, however, that workloads did not increase for the rating. The ship-wide 
manning reductions affect all rates because the workloads for in-port watches, special 
evolutions like underway replenishment, and inspection preparations are shared across all 
ship’s personnel. There is no method available to quantify any additional workload STGs 
may have taken on as a result of optimum manning, but since manning levels did not 
change significantly, the burden may have been easier to bear for this rating as opposed 
to others. 

The VAMOSC data provided revealed that training costs for the STG rating did 
decrease after the introduction of CBT. Additionally, the data showed that maintenance 
costs, particularly IM, increased over the same period. Single-factor regression analysis 
revealed no significant relationship between total training cost and the selected 
maintenance variables: Unit Level Consumption, Equipment Rework, Intermediate 
Maintenance, and Depot Level maintenance, however, examination of the scatter plot for 
IM provides some evidence to corroborate the finding that IM costs increased after CBT. 
Because the evidence was inconclusive, additional studies were used to further examine 
the cost data for selected response variables. 

Paired t-tests were performed for the following selected response variables from 
the consolidated data table: Manhours Organizational Labor, Training—NETPDTC, 
Organizational Parts Cost, Exchanges LOGCEN Cost, O&S Cost, O&S Cost minus 
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Training, and O&S Cost minus Training minus Maintenance and Modernization. The p- 
values for all response variables were less than the level of significance, 0.05, indicating 
that there is a significant difference between the means of the selected variables due to 
CBT. 

While some of the results clearly did not support the assumption that CBT would 
lead to higher O&S costs (in fact they indicated the opposite) there was evidence to 
indicate that the introduction of CBT led to higher costs in three of the response 
variables: Manhours Organizational Labor, Organizational Parts Cost and Exchanges 
LOGCEN Cost. All three categories relate to maintenance actions performed by sailors 
at the unit (ship) level. If the conversion to CBT were to have an effect anywhere in the 
Navy maintenance system, it would be at maintenance activities where sailors were 
perfonning maintenance, like afloat units and IM facilities. For this reason, rising IM 
costs noted in the VAMOSC data and scatter plots may also be attributed to the 
conversion to CBT; however, a confounding variable for IM is that several shore-based 
IM facilities were closed and shore-based IM billets for sailors were eliminated. 
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VIII. CONCLUSION AND AREAS FOR FUTURE RESEARCH 


A. CONCLUSION 

In 2001, ERNT released its report, Revolution in Training: Executive Review of 
Navy Training Final Report. The results of the report were a major overhaul in the US 
Navy’s training practices, including the replacement and supplementation of ILT with 
CBT in A and C schools. Fleet leadership reported that sailors entering the Fleet after 
receiving CBT training did not have the requisite KSATs to perform their assigned jobs. 
Extensive OJT, supervision, and assistance in performing basic maintenance tasks were 
required to bring CBT-trained sailors up to speed. 

At the same time, the U.S. Navy reorganized its maintenance program and 
introduced the optimal manning concept to the Fleet. Total manning levels on ships 
decreased and other ship requirements did not change, meaning that ships had to do more 
work with fewer personnel. Many PMS requirements were extended or eliminated on 
ships with the expectation that shore facilities would pick up the slack, but shore facilities 
experienced manning reductions at the same time. This meant that less planned 
maintenance was being perfonned on equipment, resulting in increased opportunities for 
equipment failure and decreased Fleet material readiness. 

The combination of changes in training, manning, and maintenance had negative 
effects on Fleet readiness, highlighted extensively in the Balisle report. This thesis 
examined three questions: 1) did the use of CBT in A and C schools result in sailors 
entering the fleet with lower levels of knowledge and skill sets than those sailors 
completing ILT? 2) did the transition to CBT result in cost savings in training? 3) from a 
systems perspective, if sailors trained with CBT had lower knowledge and skill levels, 
did this contribute to increased operations and maintenance costs? 

To answer the first question, government studies of the Navy’s CBT training were 
examined, which confirmed that while the transition to CBT resulted in shorter training 
times, sailors reporting to the Fleet were not as well prepared as ILT-trained sailors of the 
past. To answer the second question, DON Budget Reports from FYs 2000 to 2012 were 
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analyzed to understand changes in training costs during the periods before and after the 
implementation of CBT. The analysis reveals that overall training costs remained fairly 
flat and specialized skills training costs nearly doubled over the same period, which 
suggests that any cost savings were realized in areas other than training delivery in A and 
C schools. 

To answer the final question, the thesis examined changes to training and 
maintenance costs from both a macro and micro perspective. At the macro level, analysis 
of DON O&M budgets for the periods before and after CBT implementation shows that 
Intermediate Maintenance budgets more than doubled and Depot Level Maintenance 
budgets nearly doubled during the period of interest. Unfortunately there were too many 
confounding variables that could have affected O&M costs during this period of time to 
draw any conclusions about the effect of training on maintenance costs. To control some 
of these variables it is necessary to examine the question at a micro level. 

The thesis attempted to answer the last question by focusing on a single Navy 
system, the AN/SQQ-89(v) sonar system, to show if the results of the conversion to CBT 
are clearly connected to that system. Extensive data were collected on the 89 system to 
detennine how the impact of changes in one area, training, impacted the system as a 
whole. 

The results of the study revealed several pieces of useful infonnation. Training 
costs for the 89 did decrease from FYs 2000 to 2010 during the conversion to CBT. Over 
the same time period, maintenance costs, particularly IM costs, also increased. 
Regression analysis further indicates that a decrease in training costs is related to an 
increase in IM costs, although the relationship was not strong. Finally, paired t-tests 
showed that the conversion to CBT may have led to cost increases in Manhours 
Organizational Labor, Organizational Parts Cost and Exchanges FOGCEN Cost, all 
associated with maintenance performed by sailors at the unit (ship) level. Rising IM costs 
and increases in unit level maintenance variables due to CBT can be attributed to the fact 
that the conversion to CBT training would be most noticeable at maintenance activities 
where sailors are performing the maintenance. 
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As the Navy IG, GAO, and Balisle reports suggest, there are several factors that 
have contributed to declines in Fleet readiness. Most notably, the simultaneous 
combination of changes in training, maintenance, and manning policies appear to have 
had lasting negative impacts, including rising Fleet maintenance costs. The data analysis 
perfonned in this thesis shows that the change to CBT was statistically significant when 
compared to Operating and Support Cost variables, but it is also likely that changes to all 
three areas (training, maintenance, and manning) had collective negative effects which go 
much further than rising maintenance costs. 

The data collected and used in this analysis answered the thesis questions to a 
degree, but do not represent a complete answer. Unfortunately, answering these types of 
questions only with O&S cost data may be the only method available. Since there were 
no measures of effectiveness for training in place during the ILT era, a baseline cannot be 
established to compare effectiveness of ILT to CBT, and since there were no measures of 
effectiveness in place when CBT was introduced, it will be similarly untenable to 
measure the impact of any future changes from a training effectiveness standpoint. It is 
clear that policy changes in the 2000s impacted Fleet readiness in a negative manner, but 
no clear conclusions can be drawn about the impact of CBT on total system cost from the 
data examined in this study. 

B. AREAS FOR FUTURE RESEARCH 

It may be useful to study the impacts of the conversion to CBT on other Navy 
systems. From a training perspective, the lack of measures of effectiveness for training 
may prove frustrating in drawing any conclusions, but from a cost perspective it may be 
possible to gain further insight into the types of cost most affected by CBT. A study 
comparing the maintenance effectiveness over time of sailors trained under ILT vs. 
sailors trained primarily through CBT would be most useful, but would require a 
significant effort and a thesis that develops a system for measuring training effectiveness 
in the Navy would be also be of great interest. 
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APPENDIX : OPERATING AND SUPPORT (O&S) COST 


This appendix lists definitions for terms found in the data tables provided by 
Navy VAMOSC. All definitions were found in the VAMOSC Ships User’s Manual. 

Engineering and Technical Services - Program Office is the “cost of 
engineering and technical services provided to a ship other than during intermediate or 
depot availability. These services are provided by Navy engineering activities (Navy 
Engineering and Technical Services (NETS)) or by contractors (Contractor Engineering 
and Technical Services (GETS)).” (IBM, 2011, p. 52) 

Equipment Re-work includes the “cost of overhaul, rework, or repair of (1) 
major ordnance equipment including fire control equipment, gun mounts, torpedo tubes, 
other missile launchers, and other miscellaneous equipment.” (IBM, 2011, p. 45) 

Labor - Afloat Intermediate Maintenance is the “cost of labor expended by a 
tender, repair ship, or equivalent afloat Intennediate Maintenance Activity (IMA) for the 
repair and alteration of the indicated ship.” (IBM, 2011, p. 30) 

Labor - Ashore Intermediate Maintenance is the “Cost of labor expended by 
an ashore IMA for the repair and alteration of the indicated ship.” (IBM, 2011, p. 30) 

Maintenance and Modernization - Depot refers to “...reported costs associated 
with depot level maintenance performed for the ship by public shipyards, private 
shipyards, ship repair facilities (SRFs), and other organizations. Public-shipyard-reported 
costs come from the shipyard management information system (SYMIS), and represent 
fully burdened costs for those activities. Regional Maintenance Centers/Supervisor of 
Shipbuilding (RMCs/SUPSHIPs) -reported costs represent progress payments made by 
RMCs/SUPSHIPs to private shipyards, and are also fully burdened costs. Other cost 
elements are discussed below in their respective sections.” (IBM, 2011, p. 31) 

Maintenance - Intermediate is the “cost of material and labor expended by a 
tender, repair ship, or equivalent ashore or afloat intermediate maintenance activity in the 
repair and alteration of other vessels.” (IBM, 2011, p. 29) 
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Manhours - Intermediate Maintenance Afloat is the “labor manhours 
perfonned by a tender, repair ship, or equivalent afloat IMA for the repair and alteration 
of the indicated ship.” (IBM, 2011, p. 57) 

Manhours - Intermediate Maintenance Ashore is the “labor manhours 
perfonned by an ashore IMA on the repair and alteration of the indicated ship.” (IBM, 
2011, p. 57) 

Manhours - Organizational Corrective Maintenance is the “...manhours 
expended by the ships’ force for the performance of reported organizational corrective 
maintenance. Ships report corrective maintenance in accordance with current 
Maintenance and Material Management (3-M) System guidelines.” (IBM, 2011, p. 57) 

Material - Afloat Repair Parts Intermediate Maintenance is the “cost of 
reported repair parts used by an afloat IMA in the repair and alteration of a ship and 
installed equipment.” (IBM, 2011, p. 31) 

Material - Ashore Repair Parts Intermediate Maintenance is the “Cost of 
reported repair parts used by an ashore IMA in the repair and alteration of a ship and 
installed equipment.” (IBM, 2011, p. 31) 

Other - Depot includes all other depot-level maintenance costs not otherwise 
covered. This includes all depot maintenance at public, private, and SRF shipyards. 

Repair Parts and Repairables refer to “repairable and repair parts for use in 
maintenance of the ship and installed equipment.” (IBM, 2011, p. 26) 

Training - NETPDTC is the “cost of C, D, F, G, and T fonnal course training 
for the ship’s crew, officer and enlisted, to enable them to perform assigned maintenance 
and operational tasks.” (IBM, 2011, p. 51) 

Training - Program Office is the “cost of program office-funded training for the 
ship’s crew, officer and enlisted, to enable them to perfonn assigned maintenance and 
operational tasks.” (IBM, 2011, p. 51) 
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Unit Level Consumption is the “cost of all materials used or consumed by the 
ship with the exception of materials utilized in the intermediate and depot level 
maintenance effort.” (IBM, 2011, p. 24) 
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